monome device server

  • After a discussion about pages, multiple apps and current issues with plug and
    play, phortran and me came up with the idea of implementing a new kind of server model for the monome to automate tedious port-setup to improve the overall work?ow of all things monome.

    Think of pages’ ExternalApplicationPages combined with auto-discovery and monomeserial itself

    However, that’d need a new communication protocol, which is roughly outlined in the attached pdf.

    Also, one would need to modify the existing apps (a legacy mode that simulates the behaviour of monomeserial will be implemented as well) but we hope to develop a drop-in replacement for max msp.

    edit: there’s a bug in the forum, this is of course draft 1, not draft 12 ;)

  • this is an interesting proposal, and it definitely needs to be fleshed out before anyone sits down and tries to implement anything.

    some initial thoughts --

    (and as a side note, none of these are meant as dismissive, they're just meant to provoke thought and conversation. dissenting opinions leads to a better, more thoughtful protocol)

    the handshake may not be best suited for UDP, since packet delivery is not guaranteed. obviously if everything's on the same machine, packet delivery is pretty much guaranteed, but it's not formally so. messages lost at the handshaking stage will be detrimental to applications' functionality and likely a HUGE pain in the ass for the user to debug in a live situation.

    protocol version... version of what? why float? versions typically have multiple decimal points, and i'm still not even convinced what this means.. OSC protocol? serial protocol? perhaps an integer or a single space-delimited string (as in 1.0.4a) makes more sense?

    i also fail to see any necessary distinction between aspect name and aspect prefix.. since only one aspect may own a prefix for a given port and only one aspect may own an aspect name for a given port, why distinguish the two? is long-name and short-name really necessary?

    port should not be float. there is no port 8000.0020491.

    supported devices should include arduinome variants too.. ohm64 makes sense, since we can talk to livid about their protocol. apc40 seems like a stretch since we'll be hard pressed to get akai on the phone, and if they release a firmware update, they may brick our hard work.

    /keepalive seems well intentioned but unnecessary. again, UDP doesn't guarantee packet delivery, so if the server sends press data that never goes anywhere, it doesn't really care. ultimately, i think the /keepalive and /acknowledge will clutter the useful traffic more than /press messages to nothing.

    also, yes, if they are implemented, they should include either the app's prefix or the port that it's running on, otherwise, all you need is one /keepalive message for all open apps, which defeats the purpose.

    regarding all of the 8-bit ints in the inherited classic monome protocol -- OSC just has a few types that we are concerned with, int32, float32, and OSC-string.

    since we're not about to reinvent OSC, let's stick with 32 bit ints. helps your java too.



    "An app that supports a 40h as well as a 64 should offer additional
    functionality for tilt when connected to a 64. An app that supports only the 64
    should rely on tilt. "

    i feel like the causality here is backwards. if an app doesn't make use of tilt, why shouldn't it support both a 40h and a 64?

    if an app does make use of tilt, there should be a way for my 40h with an accelerometer to be supported too.


    i think i've touched on all of the issues that came up for me when reading, but i think the salient point about TCP vs. UDP is really important.

    you're proposing a lot of overhead that (presumably) runs behind the scenes. the added overhead could make programming for monomes much more daunting to beginners, and the question needs to be asked, what is the overhead buying you? what is your ultimate goal with this revamped server?

  • this is a fantastic idea. we were looking to add dynamic port assignment and per-device ports, but had no timeline. this would effectively serve many needs for multi-application routing.

    some comments:

    /frame seems underused, and perhaps it is in the current batch of apps, but i see very strong potential. if you have a rapid-refresh app (tiltmap should've been written this way, but chuck is clunky with matrices) it's much better to refresh on a clock using /frame instead of updating all 64 LEDs or 8 rows. the new devices have a super-optimized 9-byte serial message for updating the entire grid, which allows 30fps updating etc. otherwise the OSC/USB pipe gets clogged, which i've seen happen in maxmsp with tilt apps using rapid refresh. so i'm highly against deprecating this despite the fact it hasn't been adopted.

    part of the reason for lack of adoption is that graphical/media programming languages are really bad at using matrices. if i ever get time to write more python or VST/AU apps, i'd definitely be excited to use /frame

    regarding /clear to an on state, i personally think of "clear" as relative, given the potential that you could be coding an app for reverse colors. maybe "fill" is a more pleasing word, but then "fill" doesn't have the useful default association with "off"

    i'll keep reading and thinking.

    what environment would this be written in?

  • short answer to tehn, long answer to soundcyst still underway ;)

    if that’s the case with /frame, that’s great. Maybe we can come up with a defintion for frame for the larger devices while we are doing this

    phortran and i would prefer java for portability reasons, but i am pretty open to anything (though i am certainly most familiar with java)

  • The overall goal is eliminate the port issue and to automate the setup process as much as possible (or useful).
    I’d like to see the server being able to display a list of currently running aspects and grouping them to switch between them "pages-style"

    I see the UDP issue here, and the proposed handshaking is essentially a TCP handshake.

    I would prefer TCP as well as the use of Integers, I just don’t know how well these are supported across the board. As far as i know, Quartz Composer only accepts floats but i might be mistaken, the same goes for TCP; I’ve just seen everybody using udp-send in max msp.

    The supported devices string was meant to specify the devices the application can adapt to. If it has 64 under supported devices, it should to something with tilt, if tilt is not used for anything, the string should be "40h, 128v, 256".

    I thought of a 64 as 8x8 buttons plus tilt, thus in the context of this app, a 40h with accelerometer would be considered a 64.

    alternatively, one could let the supported devices string out and leave it to the app to react appropriately to /connect 128

    I considered aspectName to be the human readable form that contains some kind of description or the proper name of the app whereas aspectPrefix should be condensed to keep internal source code uncluttered

    for example "tintinnabulome" as aspectName and "/tintin" as aspectPrefix

    edit: just saw that there’s no 8 bit unsigned int in the OSC specification, i was pretty sure this was the case, sorry

  • Just a thought...I think you should continue to support the 40h /adc message as some users (like me!) have other stuff in addition to, or instead of an accelerometer (such as knobs). So it wouldn't really make sense to have 40h and 64 treated as one.

    Other than that, this is sounding interesting.

  • maybe we should just encode the features and dimensions instead of a device name?

  • Makes sense to me. Would probably be more future proof as other grid based devices emerge.

  • But then, how should the app tell the server which devices and features it supports?
    Because if it did, connecting a 128 to a 8x8 only app could automate the splitting of the device, quadrants-style

    If wonder if it is safe to agree upon a 8x8 segment as a kind of atomic unit?
    There are apps that can easily adapt to almost every possible size, like polygome; but how should mlr adapt to 7x5?

    How many adc ports are on a 40h? Am I right that the adc ports report 10-bit resolution data whereas the tilt on the series has only 8-bit resolution on the serial level? And the former is mapped to a float between 0.0 and 1.0 while the latter is an unsigned byte?

    Maybe we should combine tilt and adc to some kind of new, uniformed message for continuous controllers?

    It would also be possible to introduce per-device settings that map certain adc inputs to standardized osc-messages like /tiltX, /knob2 or /magnetometer0
    These could be easier communicated to the app (and its developer)

    If the user as a accelerometer, it should control delay length,
    if she has a photo-sensor, it should control delay feedback, and so on…

  • Hmmm...combining the messages sounds interesting, though to be honest I don't think it would help as every user will be different. My accelerometer is hooked up to ports 3 and 4 rather than 1 and 2 (where my knobs are)....although...maybe there could be a set of options for the user to set from within the device server which could then tell the server which message to use.

    So if you had a 40h with an accelerometer on ports 3 and 4 you could check some boxes or fill in some fields and suddenly those ports are sending a /tilt message... and maybe if you also had some knobs you could check some other boxes or something and the other ports would send /knobs or something.

    Of course the problem then becomes finding a way to combine multiple 40h messages into one... which could probably be done now that I think about it (though I don't know how well it would work).

    The other problem with merging 40h and 64 is that /tilt and /adc function quite differently. From what I can remember...

    /tilt sends a pair of floats (one for x one for Y)

    /adc sends a individual messages for each port in the form /adc int float

    When I was working on the pages tilt implementation I ended up writing seperate functions for tilt and adc as I couldn't figure any sane way of merging them.

  • that is pretty much what i had in mind

    by the way, i’m in favour for reimplementing a serial connection rather than wrapping around monomeserial (and arduinoserial, and whatnot) that would make combining the messages quite easy

  • oh...well then...in that case YES PLEASE :)

  • still we need to work that issue out somehow

    About the TCP vs. UDP issue, if Max Msp does not support TCP, writing a java or c external could solve that.

    I guess I’ll catch some sleep now…

  • Also, since it would already be setting up standardized osc-messages presumably you would be saving a preferences file that contains those mappings some how? If so, it would be very nice also include tilt calibration data preferences. I'm happy with my pages implementation but it would be nice to have it all handled in one place.

    Would probably also help getting app developers to start using /tilt more often if they didn't have to include the calibration code in their apps.

  • good idea. They could be associated to the devices serial number

  • max/msp does not, as far as i know, support TCP.

    writing an external to handle the TCP stack is probably quite a bit of trouble.

    > by the way, i’m in favour for reimplementing a serial connection rather than wrapping around monomeserial (and arduinoserial, and whatnot) that would make combining the messages quite easy

    i thought this part was a given. definitely a replacement for monomeserial, right?

    the ADC's on both the 40h and the series are 10-bit, but the series is less hackable.. i'd also seriously doubt that the device is sending a floating point number over USB.. the conversion probably happens in monome serial.

    stephen brings up a sincerely good point about the ADC's on the 40h.

    i also don't think we should have a standard unit size. i often think about being able to throw in single rows of MLR as part of a larger patch, meaning that particular module would be 1x16.

    regarding /tiltX and /knob2 type messages, these are sort of inconsistent with the rest of OSC, and according to some other folks who implement OSC, so is the OSC "dialect" that monome serial currently speaks. i had a conversation with David at Vidvox (makers of VDMX) about this a while back. he even went as far as to modify monomeserial to send messages in the format "/prefix/press/x/y/state" rather than the standard "/prefix/press x y state"


    actually, rereading david's email, here's a little bit that might be interesting (and this is related to monomeserial specifically)

    >> One other thing to mention- I noticed that you guys are using the lo
    framework for sending and receiving OSC. We've been developing our own
    OSC SDK which we've recently made Open Source. One of the new features
    that we added recently was automatic detection of other OSC sources on
    the network using Bonjour.. no need to enter an IP address or port
    information. Pretty nifty stuff, though right only a handful of other
    applications (like TouchOSC on the iPhone) support the same feature.
    Right now our SDK is only designed for iPhone & Mac, but potentially
    it could be written for Windows and Linux as well (including the
    Bonjour stuff). We are (so far) the only ones committing changes to
    the SDK, but we use it in our commercial software so there are
    frequent updates, basically whenever we find a bug (latest commit was
    a bug fix for receiving ints from the monome) or add a new feature.
    Might be of interest to some of the monome devs on the Mac? Seems a
    bit friendlier than lo, you only need to interact with the OSCManager
    object for both sending and receiving. http://github.com/mrRay/vvosc/

    i've yet to really look into it, but the email was from february, so there's about 5 or 6 months worth of work that could have happened on the project..

    in the next few weeks, i'm planning to add some features to monome serial for os x, so i'll take a look when i'm deep in it.

  • i've been doing a little reading on this whole bonjour/zeroconf/rendezvous/mDNS thing and it sounds really promising!

    some basics on it: http://en.wikipedia.org/wiki/Bonjour_(software)

    i'm no expert after 30 minutes of googling and messing around but it seems really cool. i believe the idea is that each application that implements this can register itself, and other applications can query for registered apps. it seems like no central server is required (at least from the experimenting i did) so that greatly simplifies things. also, it seems like there are already implementations for max/msp and java:

    http://opensoundcontrol.org/topic/110

    there's a link to an external here but i haven't had a chance to try it. i did try out this though:

    http://sourceforge.net/projects/jmdns/

    seems to work well, i got one of the sample apps compiled and running and it registered itself fine (i was able to verify with a viewer tool they include). if someone wants to try out the max/msp external and modify some existing app to use it (or build something simple) i'd love to experiment with the jmdns stuff and see if i can get some auto discovery code working for external apps in pages.

    anyway seems really cool and might be easily do-able with this bonjour technology.

  • What is the benefit from putting the coordinates into the Message Address Pattern?

    The jmdns worked for me, though the max external crashed quite often and i was not able to see the services the external registered from the JmDNS
    I’ll look into this

    Getting this to work with zeroconf/Bonjour would be fantastic.

    @soundcyst
    what functionality do you want to add to monomeserial?

  • ah man, too bad about the max external crashing. i wasn't able to find the source for that external but i did find the source for a pure-data oscbonjour external here:

    http://svn.puredata.info/svnroot/pure-data/trunk/externals/postlude/oscbonjour/

    can this be somewhat easily modified to work with max/msp?

    edit:

    found the max/msp external source here:

    http://osctools.svn.sourceforge.net/viewvc/osctools/trunk/oscbonjour/

    what do you guys think?

  • i’ve tinkered with this a little and downloaded an app called BonjourBrowser
    http://www.tildesoft.com/Programs.html#BonjourBrowser

    This browser picks up the services that the max patch registers but does not pick up those that JmDNS registers…

    JmDNS claims to be compatible with Rendezvous but is Bonjour (still) compatible to Rendezvous?

    The Bonjour Browser also picked up services offered by my girlfriends Ubuntu over WiFi, this could really be a great addition to the monome app development once we worked that out.
    i still don’t know how to handle the adc/tilt and device-size messages though…

  • hmm interesting, actually pretty promising news. it looks like jmDNS isn't the only library in town:

    http://www.onjava.com/pub/a/onjava/excerpt/bonjour_ch08/index.html

    there's also this bonjour browser which i'm guessing is based on the implementation described above (ie. not jmDNS):

    https://wiki.cs.columbia.edu:8443/display/res/JBonjourBrowser

    i wonder if that bonjour browser can see services registered by the max/msp external?

  • @phortran
    Yes that java based bonjour browser can see the max/msp registered services (as well as the ubuntu one)
    Check out this article http://momo.brauchtman.net/2008/12/zeroconf-made-easy-using-bonjour-for-java/

    looks really promising, i am sorry i can’t currently devote much time to this stuff as I am pretty busy with exams (that should change two weeks from now though)

    let’s not lose track on this

  • @robb

    this is excellent news, very exciting. if you or someone can build me a little test max/msp patch i'll see if i can get it auto-discovering in pages.

  • Looks like this is a go-er for python too:

    Native python implementation here:
    http://code.google.com/p/pybonjour/

    Good for ableton midi remote scripts and the likes!

  • Do not enough money to buy a house? Do not worry, just because that's achievable to receive the credit loans to resolve such problems. So get a secured loan to buy everything you want.

  • Hi,

    There has been a lot of discussion in this thread: http://post.monome.org/comments.php?DiscussionID=6443 . Many of the current conceps were discussed in a way or another. The autoconfig of monoroute popped out of this thread. Kid Sputnik has also implemented a client/server new version of monome serial.

    I would like to strongly encourage people here to join force with people there, in order to come out with a single specificiation / multiple implementation. Otherwise, we'll fall in a mess of various protocols / frameworks / routers, which will not make life of users easier.

    This is not to say that the ideas presented in this thread are not good. They do are interesting. But, what we dramatically miss at the current moment is a standard specification of all this pages / autoconfig protocols. Something that we can rely on, as we relied on the initial OSC and serial specifications that were published initially.

  • I'm with Jul, although it will take an awful log of work to get everyone on the same page. As I've stated before, the work I'm doing right now has as much to do with my own personal enjoyment/education as anything else. But I'd be happy to help combine or switch over to any other similar projects if anyone wants the help, once I at least get v1 on windows and OSX up and running( for my own sake really).

    re: handshaking - I've implemented the most basic form in my current version (/sys/handshake 'hello' or /sys/handshake 'goodbye', for opening and closing connections only). It seems like a great idea, but not quite so great in practice due to the nature of UDP. UDP feels like 2 people communicating by standing 100 feet away with blindfolds on screaming, but unfortunately TCP is just not going to happen, due to the nature of every major OSC capable client out there.

    re: /frame - I use either /frame or /led_row in every single Monome app I've ever written of any significance. so, at the very least, my Reaktor apps need that guy in order to work properly. but Reaktor is at the bottom of the list of OSC clients that will benefit from a more intelligent MonomeSerial.

    also, I agree with Soundcyst, let's please please please stick with just Int32, Float32 and String OSC data types. There are an awful lot of other types but it becomes too much. Reaktor can only deal with Float32 (float outgoing, incoming ints are converted to float), and therefore monomeserial has some hackage to convert incoming floats to ints on most messages to deal with that fact. The only other type I can see possibly being useful is the blob type, and then only if the MonomeSerial server was super extensible perhaps to end users (ive thought about this, adding some level of scripting, but that would be way down the line).

    sorry if im all over the place here, have not finished my first coffee yet =)

  • I think this thread was another victim of the random SPAM bump. Basically, this discussion started before the other one and now we do have a fairly well speced autoconfig protocol.

    @kid_sputnik
    >/sys/handshake 'hello' or /sys/handshake 'goodbye'

    Why create a new message when you can use the autoconfig one? If you use the autoconfig one then autoconfig clients can decide to connect directly to you or through an app router such as monoroute. Then each layer of the system will be compatible and symmetric.

    http://docs.monome.org/doku.php?id=tech:autoconfig

    What am I missing. Yes, we lost an hour last night and just getting my coffee.

  • Ah yes, did not look at the dates ;)

  • ahh, I didnt see that doc. I'll check that out after work.