Driving the GODOT game engine with OSC messages

Problem?

Note: I've tested this code with a very recent version of GODOT compiled from source code. Your mileage may vary.

In interactive art installations, an often used protocol to communicate between computer processes is the open sound control (OSC) protocol. Despite the name "sound" in OSC, the protocol is actually completely general purpose. In my opinion a game engine like the godot engine would benefit tremendously from built-in support for the OSC protocol because it opens up the engine to creative uses outside the gaming domain. On of my own use cases is to visualize in real-time musical events that are algorithmically generated (also in real-time) in supercollider.

The GODOT development team, despite several requests from users, chose not to support OSC directly and provide a high-level networking interface which they feel is more appropriate for their game engine (but which only works with their game engine). 

As a result, some users have attempted to write add-on modules using the GdNative C++ add-on mechanism to extend the GODOT engine. Such modules often have the drawback of limited portability and requiring users to set up a C++ building environment. In addition, the API that needs to be used by the add-ons could be changing drastically between GODOT versions. 

Therefore, I was hoping to find a more portable solution. Luckily GODOT also provides some lower level networking classes, which allow third-party developers to add some support for unsupported protocols.

OSC is typically a thin layer on top of UDP networking (sometimes TCP is used, but most often UDP is chosen for reasons of performance), and so it should be possible to add a simple OSC implementation in GDScript directly.

Note: to make sense of this blog post, you need to have a little bit of working knowledge about the GODOT game engine, at the minimum, you need to know what nodes are.

Will it perform super fast? 

Probably not. But often you don't really need to send tons of data over the network to drive your visualization. Think about optimizing your data streams.

Will it be easy to use?

I think it is extremely easy to use. But feel free to try it out and judge for yourself.

How can I use it?

Glad you ask. Here's the big picture:

To the root note of our GODOT sketch (usually a Node2D for 2d games, or a Spatial for 3d games), I attach a script that monitors the network for incoming OSC messages. The OSC messages are decoded and automatically dispatched to child nodes who requested to be kept informed about certain incoming messages.

Consider an example of a Node2D rootnode (which I renamed to RootNode in the IDE) with a Sprite child node. In the script attached to the Sprite, I only have to express my desire to be kept up-to-date about incoming OSC messages, e.g. here's the complete script for a sprite that can change position by receiving a /set/xy OSC message.

extends Sprite

func _ready():
   get_tree().get_root().get_node("RootNode").register_dual_arg_callback("/set/xy", get_node("."), "set_xy")

func set_xy(x, y):
   self.position.x = x
   self.position.y = y

This means that when the Sprite is instantiated, it registers itself with the root node ("register_dual_arg_callback"). It tells the root note that it wants its function "set_xy" to be called whenever a "/set/xy" OSC message is received over the network with two arguments (x and y) which in this example then are used to set the sprite position. There's also a register_single_arg_callback for functions that expect only one value (and nothing prevents you from adding more variants).

This simple code suffices to ensure that from now on, this particular Sprite will automatically react to incoming OSC /set/xy messages by updating its position.

E.g. in an environment that supports sending OSC message like supercollider, I can run the following snippet to update the sprite position:

(

b = NetAddr.new("127.0.0.1", 4242); // create the NetAddr
b.sendMsg("/set/xy", 650, 200);

)

The godot sketch automatically reacts to the OSC message by updating the sprite xy position to 650, 200, exactly what we requested the system to do by registering a callback. Needless to say the OSC message could perfectly come from another computer, tablet or phone (e.g. from the TouchOsc app or from Open Stage Control).

The value 4242 is the port number on which the root node is listening for incoming OSC messages. In the proof of concept code it's hardcoded to 4242, but you are of course free to change it or to make it configurable.

Ahm, ok. Show me this magic code that goes into the root node then?

Note: the code in this section is obsolete, and replaced by the code in part 2 of this article: https://technogems.blogspot.com/2021/11/driving-godot-game-engine-with-osc.html

Well... it's not the prettiest code (it's a proof of concept - things like IP address and port are hardcoded, but should be simple to change). It works well enough for me at the moment (but I've only done some basic experiments so far). In case I run into trouble, I may need to revisit the details. If you run into trouble, please let me know what happened and how you solved it (or just explain the problem and we can discuss to see if we can find a solution).

Here's the code I put in the root node. For now it only supports parsing integer, float, string and blob OSC messages. In the future maybe also OSC bundles and some other data types could be added, but even with only this simple subset of OSC supported, the possibilities are already endless.

Note that the call to OS.set_low_processor_usage_mode may not be portable across all platforms, but it's probably not strictly needed to make the system work (but don't take my word for it, I'm not at all experienced with GODOT on platforms other than linux).

The code for registering callbacks and dispatching probably can be made a bit more general, and there could be support for pausing, resuming and stopping reacting to OSC notifications but for demonstration purposes what I have here should suffice.

extends Node2D

var IP_CLIENT
var PORT_CLIENT
var PORT_SERVER = 4242 # change me if you like!
var IP_SERVER = "127.0.0.1" # change me if you like!
var socketUDP = PacketPeerUDP.new()
var observers = Dictionary()

func register_single_arg_callback(oscaddress, node, functionname):
 observers[oscaddress] = [1, node, functionname]
 
func register_dual_arg_callback(oscaddress, node, functionname):
 observers[oscaddress] = [2, node, functionname]
 
func _ready():
 OS.set_low_processor_usage_mode(true)
 start_server()
 
func all_zeros(lst):
 if lst == []:
  return true
 for el in lst:
  if el != 0:
   return false
 return true

func _process(_delta):
 if socketUDP.get_available_packet_count() > 0:
  var array_bytes = socketUDP.get_packet()
  #var IP_CLIENT = socketUDP.get_packet_ip()
  #var PORT_CLIENT = socketUDP.get_packet_port()
  var stream = StreamPeerBuffer.new()
  stream.set_data_array(array_bytes)
  stream.set_big_endian(true)
  var address_finished = false
  var type_finished = false
  var address = ""
  var type = ""

                # parse osc address
  while not address_finished:
   for _i in range(4):
    var addrpart = stream.get_u8()
    if addrpart != 0:
     address += char(addrpart)
    if addrpart == 0:
     address_finished = true
    

                # parse osc type list 
  while not type_finished:
   for _i in range(4):
    var c = stream.get_u8()
    if c != 0 and char(c) != ",":
     type += char(c)
    if c == 0:
     type_finished = true

  # decode values from the stream
  var values = []
  for type_id in type:
   if type_id == "i":
    var intval = stream.get_32()
    values.append(intval)
   elif type_id == "f":
    var floatval = stream.get_float()
    values.append(floatval)
   elif type_id == "s":
    var stringval = ""
    var string_finished = false
    while not string_finished:
     for _i in range(4):
      var ch = stream.get_u8()
      if ch != 0:
       stringval += char(ch)
      else:
       string_finished = true
    values.append(stringval)
   elif type_id == "b":
    var data = []
    var count = stream.get_u32()
    var idx = 0
    var blob_finished = false
    while not blob_finished:
     for _i in range(4):
      var ch = stream.get_u8()
      if idx < count:
       data.append(ch)
      idx += 1
      if idx >= count:
       blob_finished = true
    values.append(data)
   else:
    printt("type " + type_id +" not yet supported")

  if observers.has(address):
   var observer = observers[address]
   var number_args = observer[0]
   var nodepath = observer[1]
   var funcname = observer[2]
   if number_args == 1:
    nodepath.call(funcname, values[0])
   elif number_args == 2:
    nodepath.call(funcname, values[0], values[1])
    
func start_server():
 if (socketUDP.listen(PORT_SERVER) != OK):
  printt("Error listening on port: " + str(PORT_SERVER))
 else:
  printt("Listening on port: " + str(PORT_SERVER))

func _exit_tree():
 socketUDP.close()

Compared to the misery of adding, compiling, maintaining, porting, ... a GdNative module, I think this is pretty acceptable (at least for my use cases).