can we talk about the problem with serialosc?

  • ..namely the cpu use problem and incompatibility with arduino/other serial devices. After reading several older threads about these issues, I would like to rekindle the conversation - particularly because serialosc refuses to play nicely with more than just arduino devices.

    For instance, if I plug in my vavdo controller, I get the same problems... serialosc cpu goes to 100% and I get kernel panics if devices are unplugged. Yes, I have learned to stop and start serialosc with commands in terminal before plugging/unplugging devices, but that's not really the point.. and it doesn't address the cpu use problem.

    I can appreciate that serialosc runs ubiquitously in the background, allowing for multiple monomes and arcs to be connected. Very cool. Makes me want to get more than one monome. But apparently if I am not plugging in a monome or an arc, there are problems. The problem isn't made any better by the fact that serialosc automatically starts on reboot and there is no way to stop that process. So what I have is software that is designed to run at a low level with 'high level functionality' that freezes other devices, jacks cpu, causes kernel crashes and can't be prevented from starting on bootup.

    The suggestion to 'stop and start serialosc' doesn't work. And (forgive the tone..) blaming ftdi does't work either. These problems are mentioned on threads dating back more than a year.. with assurances that it's 'being worked on'. That's cool.. I know some things take time. I'm just kinda wondering.. what's up? I recently picked up a brand new monome256 that I can't use with my custom instruments... which pretty much sucks, because, well, that's why I got it.

    Thanks for reading and I hope you can understand my frustration with this matter... I have waited to post about it because I wanted to make sure that I exhausted all other resources before asking about it. I deeply appreciate this community and all of the efforts that have gone into creating this platform.. surely we can resolve this.

  • i understand your frustrations and indeed this issue is a large reason i haven't spent more time with the arduino system. i'm not really qualified to look at C code at all, but i'm going to start having a sift through the code and see if anything sticks out to someone who hasn't looked at the code for 100s of hours. agreed that this has probably taken longer than it should have - that being said, i'm unaware of how deeply this bug is intwined in the codebase.

    as an interim solution i believe plugging in the monome via a usb hub can stop the kernel panics when plugging in/out, though i don't think it solves the CPU issue (i'm aware this is non-ideal and can mean a whole extra power plug).

    finally, i might suggest you change the subject name of the post to something more explicit: eg. "let's fix the serialosc CPU problem (arduino incompatibility)".. might gather a more directed response to solving the problem.

  • "can we talk about the problem with serialosc?"

    First rule of serialosc

    Second rule of serialosc

    That-which-must-not-be-named has evaded the judgment of the Monomistry Of Magic for centuries. And now fear of the name has caused fear of the thing itself! Beware! The Dark Lord Approacheth!

    What is his name now?
    oracles is
    recoils as
    solace sir
    caress oil
    scares oil
    across lie
    recoil ass
    slices oar
    closes air
    scores ail
    clears so i
    scales or i
    relics so a
    slicer so a
    slices or a
    closer as i
    closer is a
    acres oils
    acres silo
    acres soil
    cares oils
    cares silo
    cares soil
    races oils
    races silo
    races soil
    scare oils
    scare silo
    scare soil
    coals ires
    coals rise
    coals sire
    sails core
    solar ices
    lasso rice
    soars lice
    slice oars
    slice soar
    cries also
    close airs
    close sari
    cores sail
    score sail
    coils ears
    coils eras
    coils sear
    rises coal
    sires coal
    soils acre
    soils care
    soils race
    class ore i
    class roe i
    close sir a
    cross ale i

    no! all but dark magic in anagrams! It's name is SERIALOSC!!!!! *flee for your lives!*

    haha, ok ok, i had my fun, i'll shut up now.

  • Personally I install manually with homebrew

    brew install serialosc

    It doesn't run at start up, and I launch it manually. There is a luanch daemon for serialosc that can be removed as well... just forget the exact way to do it atm

  • From other thread:

    on osx

    that is the file that makes it launch

    launchctl stop org.monome.serialosc
    launchctl start org.monome.serialosc"

  • @ galapagoose - I didn't know that about the usb hub. I was actually avoiding using one.. i'll try it. and thank you very much for looking into this. it's crazy that the standard monome operating software is incompatible with arduino. unbelievable really. that is not a minor issue. it's like a toaster oven that's incompatible with pizza or a universal adapter that doesn't do 9v. we can call the monome minimalist. we can call it an interface. but adaptable? yeah.. in the little town of monome maybe. but what about the rest of the world?

    @karaokaze - ok. i laughed. ha. and yes.. I am very interested in the max option. I love max. I actually enjoy solving problems in max.. so if you have any leads or resources to communicate directly with the monome via usb, I would be very grateful. I take it that osc goes out the window if I set up my own serial connection? is it reasonably possible for me to create abstractions to convert osc addresses and values into serial messages so that once everything is set up, the actual programming can be done in osc format?

    @thealphanerd - thank you. I knew there was a way

    over all my biggest concern is getting other serial devices to function with the monome while using osc to control everything... and to stop the cpu from getting jacked.

  • I also have a second computer. Perhaps it would be easier for me to connect my other serial device hardware on that system. That way, there would be no serial conflicts on either computer. all of the devices could play together via udp. definitely _not_ ideal.. but perhaps easier than trying to create my own serial to osc converter using standard max objects..

  • also, the CPU bug is totally frustrating, but it's not really '100%' — it's 100% of one core. if you've got a 'dual-core' computer it'll hyperthread to create 4 virtual cores, so in essence it only sends the CPU to 25%... not great, but not a game breaker.

    i know it's also not perfect, but if you use a USB-integrated arduino (ie. leonardo or due) you won't have the CPU issue.

  • honestly i'm embarrassed that the problem isn't fixed and i deeply apologize. i'm still learning how to properly delegate tasks and in the meantime things slip through the cracks.

    right now, at this very moment, i will try to set this fix in motion.

  • For me the biggest issue is that it's not possible to use a monome and any kind of arduino-based device at the same time. Serialosc blows up (100% cpu) and the Arduino stuff drags to a crawl.

    So far I've not needed to perform with both at the same time, but I have been developing a setup based around using both.

  • "the CPU bug is totally frustrating, but it's not really '100%' — it's 100% of one core. if you've got a 'dual-core' computer it'll hyperthread to create 4 virtual cores, so in essence it only sends the CPU to 25%... not great, but not a game breaker."

    So my single core Athlon XP is not going be cutting it anymore? Damn!

    (Not a joke. That's what I'm running. It's why I'm still using monomeserial.)

  • visinin says he's fixed the issue-- he's sending me build instructions.

  • @rajaokaze: Clearly I was confused. Thanks for setting me straight. Although, I have had CPU spiking issues on my ancient machine when running Max and mlrV2 even with my gs128, but that's a problem with me being too cheap to upgrade. My brother did just bring me a Dell Optiplex tower that his company was sending to the recycle bin, and it's got a 3.2 GHz P4 and 4 gigs of RAM, so I'm about to party like it's 2007!

    And you are totally right about the arduinome thing. Hell, I can't even get around to mowing my lawn, so yeah, anything "project" based--which is just code for "some complicated set of tasks that I'm putting off until later"--ain't gonna happen.

    Plus, change scares me. This new-fangled serialOSC all the kids are using these days, what's that all about?

  • @tehn - ok.. now that is impressive.

  • As someone who is getting into arduino this very moment, thank you very much for bumping this metanome!

  • *pokes head in* did someone say problem fixed...and build instructions are being sent?

  • well, i have the thing built. need to pack a new installer (waiting to get the build script.)

    tested good with my ftdi serial dongle. no cpu spike.

  • tehn, that's great news... thanks for taking care of this. looking forward to the update.

  • also on the wiki and setup guide.

  • Hey Tehn, I'm having trouble installing this verison of serialosc. I have a newer laptop that hasn't had any previous versions of serialosc installed on it yet. I get a message at the end of the installation process saying "Installation Failed: The installer encountered an error that caused the installation to fail"

    I'm running on a macbook pro, intel core i7 on OS 10.8.4.

    Looking to get this laptop up and running with the monome now. Thanks!

  • Cool. Will install this evening and report back!

    Question: does this fix address the kernel panic issue on disconnect?

  • @egon: ack, this was built on 10.7.5, i'm not sure that makes a difference. try 1.2a:

    @emergencyofstate: i'm uncertain, i'll ask will. do you experience this often?

  • whoa, 10.8.4 is only $19, i guess i'll upgrade...

  • @tehn Yup..So much to where I cringe every time I unplug my device cause I'm just dreading that slow grey screen of death that comes down vertically from the top of the screen.

    Probably happens about 1/10 for me (had one last night).

    I dug up these old threads on the subject.

    Although in the second thread there, you mention it's a FTDI code bug and you had sent off a logic board and they were attempting to fix..

    Is this all the same issue?

  • Ah, thanks tehn!

  • @tehn - It installed without a hitch...I was able to run an ftdi device + monome with no cpu problems... will check arduino devices later. will also test the kernel panic problem and report back. thanks again!

    oh... and this is all on a mbp core i7 running 10.8.4.

    @egon77 - I'm running the same gear and I installed it with no problem... not sure if it matters but I quit all major apps as well as serialosc and ftdi (via terminal) before installing.

  • @karaokaze - quitting serialosc and ftdi also seems to stop the kernel panic problem. In fact, I'm pretty sure I haven't had any crashes quitting serialosc alone before unplugging.. either way, I have had the impression that the problem was the way serialosc was communicating with ftdi ,not the other way around... could be wrong.

  • i believe the issue is not limited to serialosc, but having any program working with an open ftdi port when the plug is pulled. i suspect there's a workaround. definitely stopping serialosc prior to unplug will prevent a crash.

    that said, i potentially plug/unplug more monomes (testing/building) than anyone, and i have not had this crash since i can remember. 2011 macbook air on 10.7.5.

  • @egon77 does it mention a log file you can check for the error?

  • I don't remember seeing any mention of a log file. I can attempt again when I get home. I did install 1.2 without a hitch. Should I stop serialosc via terminal before re-attempting 1.3?

  • you shouldn't need to stop serialosc.

    also-- the only thing 1.3 updates is allowing non-monome ftdi devices not to peg the cpu. which is big for many people, but not for others.

  • @tehn "that said, i potentially plug/unplug more monomes (testing/building) than anyone, and i have not had this crash since i can remember. 2011 macbook air on 10.7.5."

    This tells me there has to be some other factor then. I'm on a 2.2 Ghz 10.7.5 MacbookPro and it's very consistent for me. In fact, it happened with my last macbook pro as well, and with both recent versions of serialosc and monomeserial before that!

    Only thought I have atm is Maxforlive. It may somehow compound this problem? It's the one constant I've had throughout that may be different from yours?

    Anyway - I'll not pollute this thread with this specific issue. This fix is super cool and will have some substantial positive impact on the community as a whole.

    Thank you Will and Brian!

  • Perhaps something in the 1.2 installer provided support for the install of 1.3 because I just tried installing 1.3 and it was successful. Again, this laptop was new to serialosc altogether when I first tried the 1.3 installer. So maybe that was a factor. Thanks for the help!

  • @egon77: there is a chance that something is missing from the 1.3 installer-- i'll have to clean off a system for a test install. thanks for the tip.

    regarding maxforlive, it's separated by an OSC layer which to me shouldn't have an effect. i apologize this issue has dragged out forever.

  • regarding the FTDI crashes: as far as I can tell, they're happening in the official FTDI driver and FTDI just doesn't care enough to actually fix the bug(s). we've been bugging them about it for years.

    the issue happens only on OSX, when a program has an FTDI serial device open and the device is unplugged. now, serialosc handles this internally and cleans up the OSC server for that device, but something is amiss in the kernel driver. the best i can determine is that it's a class of bugs called a "race condition" -- essentially, a bug due to a lack of synchronization. i think that, when a device gets unplugged, the driver frees some data structures associated with it, but then some other piece of the driver attempts to access something from the recently-freed structure(s), and everything blows up.

    it's (probably) not just monome devices, and it's certainly not just serialosc. when i was originally developing serialosc, i reproduced the crash many, many times with monomeserial, but since you can quit monomeserial (and it seems people often did when they were finished with max/msp), the panic cropped up less often.

    the way you get around it is by having serialosc not running when you unplug a device, either by stopping it manually or by logging out (since serialosc is a per-user process).

    it may be possible to prevent this from happening by essentially making the OSC server process drop everything in a "not my circus, not my monkeys" style, not even trying to clean up the open monome device, but that seems like sketchy programming practice for covering up a lazy company's lack of effort in fixing their fucking kernel driver (yeah FTDI i'm calling you out).

  • google "ftdi kernel panic" and you'll see that we're only ranked 2 and 3 for hits. many many others experiencing this.

    vis is correct, though. if you are experiencing this a lot, best practice would be to stop serialosc prior to disconnecting. there are clickable scripts on the wiki if you don't want to logout fully.

  • technical deets:

    looked through the serialosc code a bit to refresh my memory. the technical deets are that the serialosc OSC server process spends most of its time in select() (on OSX), listening on both the serial device and the OSC server. if we receive an error on the serial device, that means it's been disconnected, and the process bails out. on the way out though, it calls close() on the serial device, which i think is where the panic happens. the race condition would be that the FTDI driver cleans up the device when it goes away, returns the error condition to select(), but the close() call causes the driver to reference previously-cleaned-up memory, and there's a panic.

    i could theoretically omit the close() call on OSX, but i believe that processes which exit with open file descriptors have them cleaned up by the kernel (i.e. they get close()d anyway). this line of thinking is what i describe by "sketchy programming practice" because we're avoiding cleaning up a resource (serial device) just so we can cross our fingers and hope the FTDI driver doesn't pitch a fit.

  • ^0o^ Thanks so much for the detailed assessment of the issue. I will head your suggestions for best practices.

    I can't help think it seems a bit hopeless considering you've been bugging FTDI about this for years -- I'd have to imagine others have been too (looks like this bug affects Linux users too). However, another part of me wonders what kind of pressure we can put on them to just fuckin-fix-your-bug already.

  • @karaokaze - I suspect you may be slightly crazy - but it may also contribute to your genius so don't change anything.

    Being that the point is not to have serialosc running, I opt to turn it off. I have the off-serialosc.command and on-serialosc.command shortcuts in my sidebar. It's actually faster and easier for me to turn off serialosc than it is to logout because I don't have to log back in... and I don't interrupt other processes associated with my account.

  • ^^ I'm gonna go the terminal command shortcuts because I'm using my work laptop currently and when at home (not on my work domain) whenever I try to log out/in it does this 60 second, try to connect to my work domain bullshiz.

  • i may make an experimental build with the bad-programming-practice...

  • @tehn
    serialosc/src/serialosc.c, line 95. try commenting out the call to "monome_close()".
    no guarantees, though. i suspect that file descriptors that are open when a process exits will be closed by the kernel as it's cleaning up the process.

    still, maybe worth a shot. good luck.

  • @emergencyofstate - it's sooo much easier... I also modified the .command files with the additional line:

    osascript -e 'tell application "Terminal" to quit'

    and changed the shell preferences in terminal to close the window if the shell exits cleanly (so I know it actually executed properly) and to never to prompt me when closing. the entire process takes about a quarter of a second and doesn't leave terminal running.

    of course.. those preferences in terminal are global, so you may prefer to be prompted before quitting if you use terminal a lot.. then again, if you use terminal a lot, you probably know a better solution anyway.