Answered
1
0

How can I go about syncing game events to beats in a song?

Ideally I could set a marker in FMOD at every beat I want something to happen (Say color on a unity object shifts from black to white on every other beat). Then create a script in Unity that listens for the marker?

Maybe this could also be done with automating a parameter? Could you could read what the parameter is set at for each frame?

I’d like to easily tailor where these shifts happen within FMOD.

If this isn’t possible, what’s a sound way to go about syncing game events to a beat?
Any help would be appreciated. Thanks!

  • You must to post comments
Best Answer
1
1

If you set a tempo marker on the timeline in Studio then markers are automatically quantized to the beat (hold ctrl to dragging a marker to disable quantization).

If you want to get every beat, simply use the beat callback.

public class MarkerDemo : MonoBehaviour 
{
    FMOD.Studio.EventInstance instance;
    FMOD.Studio.EVENT_CALLBACK cb;

    void Start()
    {
        instance = FMOD_StudioSystem.instance.GetEvent("event:/Music/Complex/Situation_Oriental");


        cb = new FMOD.Studio.EVENT_CALLBACK(StudioEventCallback);
        instance.setCallback(cb, FMOD.Studio.EVENT_CALLBACK_TYPE.TIMELINE_MARKER | FMOD.Studio.EVENT_CALLBACK_TYPE.TIMELINE_BEAT);
        instance.start();
    }

    public FMOD.RESULT StudioEventCallback(FMOD.Studio.EVENT_CALLBACK_TYPE type, IntPtr eventInstance, IntPtr parameters)
    {
        if (type == FMOD.Studio.EVENT_CALLBACK_TYPE.TIMELINE_MARKER)
        {
            FMOD.Studio.TIMELINE_MARKER_PROPERTIES marker = (FMOD.Studio.TIMELINE_MARKER_PROPERTIES)Marshal.PtrToStructure(parameters, typeof(FMOD.Studio.TIMELINE_MARKER_PROPERTIES));
            IntPtr namePtr = marker.name;
            int nameLen = 0;
            while (Marshal.ReadByte(namePtr, nameLen) != 0) ++nameLen;
            byte[] buffer = new byte[nameLen];
            Marshal.Copy(namePtr, buffer, 0, buffer.Length);
            string name = Encoding.UTF8.GetString(buffer, 0, nameLen);
            if (name == "HIGH")
            {
                UnityEngine.Debug.Log("Reached high intensity marker");
            }
        }
        if (type == FMOD.Studio.EVENT_CALLBACK_TYPE.TIMELINE_BEAT)
        {
            FMOD.Studio.TIMELINE_BEAT_PROPERTIES beat = (FMOD.Studio.TIMELINE_BEAT_PROPERTIES)Marshal.PtrToStructure(parameters, typeof(FMOD.Studio.TIMELINE_BEAT_PROPERTIES));
        }
        return FMOD.RESULT.OK;
    }
}
  • Chris Eng

    Thanks for sharing this snippet, it helped me understand how to implement some beat-driven features in my own Unity project.

    If I may ask a follow-up question, what kind of latency should we expect when receiving these callbacks, particularly the TIMELINE_BEAT callbacks? When using MonoBehaviour.Update to call FMOD.Studio.System.Update, I measure TIMELINE_BEAT callbacks occurring up to ±0.02s sooner/later than expected, and ±0.01s on average. When using MonoBehaviour.FixedUpdate instead, I still measure callbacks occurring up to ±0.02s sooner/later than expected, but ±0.00s on average. I am using UnityEngine.Time.realtimeSinceStartup to record timestamps.

    These slight variations in period between callbacks are generally negligible for non-gameplay-critical features like beat-driven effects, but seem to be significant enough to cause trouble when trying to implement beat-driven input–sometimes input will be rejected because a callback occurred a little sooner or a little later than usual, for example.

  • Geoff Carlton

    The timeline processing occurs on the Studio thread once per Studio update (every 20ms). The beat callback is deferred for the main Unity update which will add some floating delay which is probably what you have seen.

    It would be a nice feature for the callback structure to include some timing information that includes that delay, which would allow very accurate timing, but we don’t have any plans to implement that feature right now.

  • Harley

    Hi, i’m rather new to unity and fmod integration, me being lazy was hoping nicholas wilcox’s snippet of code would build , however i am having a few problems setting up the code, initally i changed the line FMOD_StudioSystem.instance.GetEvent/ to FMODUnity.RuntimeManager.CreateInstance(“event—
    which solved an error, however IntPtr namePtr = marker.name; is throwing up an error, “cannot implicitly convert type FMOD.StringWrapper to system.IntPtr. I’m unsure as to the issue and how it could be resolved? any advice would be greatly appreciated

  • Nicholas Wilcox

    There’s a sample in the docs for the new integration regarding timeline callbacks: http://www.fmod.org/documentation/#content/generated/engine_new_unity/script_example_timeline.html

  • You must to post comments
0
0

Thank you so much!! Works perfectly.

Note:
I had to add a few namespaces:

using UnityEngine;
using System.Collections;
using System;
using System.Runtime.InteropServices;
using System.Text;

  • You must to post comments
Showing 2 results
Your Answer

Please first to submit.