• Queues vs. Sockets

    From MCMLXXIX@MDJ to Digital Man on Monday, April 28, 2008 21:37:53
    Ok.. now that I've cleared up the issues surround my ability to even ask questions in here...

    I've come really far with some of these javascript games... and the next logical step is making them multiplayer (real-time)

    With the amount of data I intend to be passing back and forth between nodes, file IO seems like a huge PITA. I could be wrong, but that's just how it seems. I've had moderate success using named queues to communicate between nodes in a makeshift "chat lobby" I made... but it leaves open Queues behind in memory, which interferes with the ability to detect whether or not there are any other users connected.

    that leaves me with sockets, unless there's a way to free up queues from memory so they don't show up in the named_queue list.

    that being said.. is there a simple way to create a local socket connection between 2 nodes? the format is a little confusing, and I've had no success so far making it work.

    thanks for your patience (with my fumbles)


    ---
    ■ Synchronet ■ My Brand-New BBS
  • From Deuce@SYNCNIX to MCMLXXIX on Tuesday, April 29, 2008 00:36:07
    Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Mon Apr 28 2008 09:37 pm

    With the amount of data I intend to be passing back and forth between
    nodes, file IO seems like a huge PITA. I could be wrong, but that's just
    how it seems. I've had moderate success using named queues to communicate between nodes in a makeshift "chat lobby" I made... but it leaves open Queues behind in memory, which interferes with the ability to detect
    whether or not there are any other users connected.

    File I/O is most likely to work...

    that leaves me with sockets, unless there's a way to free up queues from memory so they don't show up in the named_queue list.

    that being said.. is there a simple way to create a local socket connection between 2 nodes? the format is a little confusing, and I've had no success so far making it work.

    Yes, you have one node bind/listen on a port and another node connect to it. But it sounds like what you want is many nodes connected to many other nodes. If reliability isn't needed AND it doesn't need to work on multisystem installs, slicing off a part of the localnet and using broadcase would most likely be easiest. Basically, have each node bind to a known port on 127.0.0.X where X is the node number. I think XP has an issue with this though... see: http://support.microsoft.com/default.aspx?scid=kb;[LN];884020

    I'll see what I can whip up quickly for an example...

    ---
    Synchronet - Jump on the Web 0.2 bandwagon!

    ---
    ■ Synchronet ■ My Brand-New BBS (All the cool SysOps run STOCK!)
  • From Deuce@SYNCNIX to MCMLXXIX on Tuesday, April 29, 2008 01:07:28
    Re: Queues vs. Sockets
    By: Deuce to MCMLXXIX on Tue Apr 29 2008 12:36 am

    Yes, you have one node bind/listen on a port and another node connect to
    it. But it sounds like what you want is many nodes connected to many other nodes. If reliability isn't needed AND it doesn't need to work on multisystem installs, slicing off a part of the localnet and using
    broadcase would most likely be easiest. Basically, have each node bind to
    a known port on 127.0.0.X where X is the node number. I think XP has an issue with this though... see: http://support.microsoft.com/default.aspx?scid=kb;[LN];884020

    I'll see what I can whip up quickly for an example...

    load("sbbsdefs.js");
    load("sockdefs.js");

    var s=new Socket(SOCK_DGRAM, "InterNode");
    if(!s.bind(5000, "127.0.0."+bbs.node_num)) {
    writeln("Bind failed: "+s.error);
    exit(1);
    }
    if(!s.listen()) {
    writeln("Listen failed: "+s.error);
    exit(1);
    }
    writeln("Q to quit or # to send message to node");

    while(1) {
    var k=console.getkey(K_NONE, 100);
    if(k.toUpperCase()=='Q')
    break;
    if(parseInt(k) > 0) {
    console.write(": ");
    var msg=console.getstr("", 77);
    s.sendto(msg,"127.0.0."+k, 5000);
    }
    if(s.data_waiting) {
    var d;
    d=s.recvfrom();
    writeln("Message from "+d.ip_address);
    writeln(d.data);
    writeln("");
    }
    writeln("Loop!");
    }


    This does NOT work under FreeBSD since you cannot actually listen on any loopback address except 127.0.0.1... not sure of other OSs. Failing that, you can use port number X+bbs.node_num and just not be able to broadcast... which may be enough. Fiddle with the 5000 instead of 127.0.0. in the script.

    TCP gets trickier. :-)

    ---
    Synchronet - Jump on the Web 0.2 bandwagon!

    ---
    ■ Synchronet ■ My Brand-New BBS (All the cool SysOps run STOCK!)
  • From MCMLXXIX@MDJ to Deuce on Tuesday, April 29, 2008 08:26:38
    Re: Queues vs. Sockets
    By: Deuce to MCMLXXIX on Tue Apr 29 2008 00:36:07

    Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Mon Apr 28 2008 09:37 pm

    With the amount of data I intend to be passing back and forth between nodes, file IO seems like a huge PITA. I could be wrong, but that's just how it seems. I've had moderate success using named queues to communicate between nodes in a makeshift "chat lobby" I made... but it leaves open Queues behind in memory, which interferes with the ability to detect whether or not there are any other users connected.

    File I/O is most likely to work...

    that leaves me with sockets, unless there's a way to free up queues from memory so they don't show up in the named_queue list.

    that being said.. is there a simple way to create a local socket connecti between 2 nodes? the format is a little confusing, and I've had no succes so far making it work.

    Yes, you have one node bind/listen on a port and another node connect to it. But it sounds like what you want is many nodes connected to many other nodes If reliability isn't needed AND it doesn't need to work on multisystem installs, slicing off a part of the localnet and using broadcase would most likely be easiest. Basically, have each node bind to a known port on 127.0. where X is the node number. I think XP has an issue with this though... see http://support.microsoft.com/default.aspx?scid=kb;[LN];884020

    I'll see what I can whip up quickly for an example...


    I pretty much have no idea what you're talking about.. Guess I've got some investigating to do. and don't make fun of my tagline! I had just reset my message bases!

    ---
    ■ Synchronet ■ The BRoKEN BuBBLE (MDJ.ATH.CX)
  • From MCMLXXIX@MDJ to Deuce on Tuesday, April 29, 2008 08:39:08
    Re: Queues vs. Sockets
    By: Deuce to MCMLXXIX on Tue Apr 29 2008 01:07:28

    Re: Queues vs. Sockets
    By: Deuce to MCMLXXIX on Tue Apr 29 2008 12:36 am

    Yes, you have one node bind/listen on a port and another node connect to it. But it sounds like what you want is many nodes connected to many othe nodes. If reliability isn't needed AND it doesn't need to work on multisystem installs, slicing off a part of the localnet and using broadcase would most likely be easiest. Basically, have each node bind t a known port on 127.0.0.X where X is the node number. I think XP has an issue with this though... see: http://support.microsoft.com/default.aspx?scid=kb;[LN];884020

    I'll see what I can whip up quickly for an example...

    load("sbbsdefs.js");
    load("sockdefs.js");

    var s=new Socket(SOCK_DGRAM, "InterNode");
    if(!s.bind(5000, "127.0.0."+bbs.node_num)) {
    writeln("Bind failed: "+s.error);
    exit(1);
    }
    if(!s.listen()) {
    writeln("Listen failed: "+s.error);
    exit(1);
    }
    writeln("Q to quit or # to send message to node");

    while(1) {
    var k=console.getkey(K_NONE, 100);
    if(k.toUpperCase()=='Q')
    break;
    if(parseInt(k) > 0) {
    console.write(": ");
    var msg=console.getstr("", 77);
    s.sendto(msg,"127.0.0."+k, 5000);
    }
    if(s.data_waiting) {
    var d;
    d=s.recvfrom();
    writeln("Message from "+d.ip_address);
    writeln(d.data);
    writeln("");
    }
    writeln("Loop!");
    }


    This does NOT work under FreeBSD since you cannot actually listen on any loopback address except 127.0.0.1... not sure of other OSs. Failing that, y can use port number X+bbs.node_num and just not be able to broadcast... whic may be enough. Fiddle with the 5000 instead of 127.0.0. in the script.

    TCP gets trickier. :-)


    Ok that makes sense. I'll see what I can do with it. Thanks!

    ---
    ■ Synchronet ■ The BRoKEN BuBBLE (MDJ.ATH.CX)
  • From Lord Time@TIME to MCMLXXIX on Tuesday, April 29, 2008 10:48:00

    ---
    ■ Synchronet ■ My Brand-New BBS

    Please goto your scfg-networking-qwk networking-default tagline, and put in
    the name of your bbs & telnet address

    ---
    Rob Starr
    Lord Time SysOp of Time Warp of the Future BBS
    Telnet://Time.Darktech.Org:24
    or Telnet://Time.Synchro.Net:24
    ICQ # 11868133 Yahoo : lordtime2000
    AIM : LordTime20000 MSN : Lord Time

    ■ CMPQwk 1.42 16554 ■ Upload Error #69 - Blonde Not Attached
    ---
    ■ Synchronet ■ Time Warp of the Future BBS - Home of League 10 IBBS Games
  • From Angus McLeod@ANJO to Deuce on Tuesday, April 29, 2008 09:04:00
    Re: Queues vs. Sockets
    By: Deuce to MCMLXXIX on Tue Apr 29 2008 00:36:00

    Yes, you have one node bind/listen on a port and another node connect to it. But it sounds like what you want is many nodes connected to many other nodes

    How about a comms-handler task that queues all the data and manages distribution by node number?

    ---
    Amarok: 17,459 tracks from 1,238 albums by 1,349 artists, but none playing.
    ■ Synchronet ■ Making sure Jason works OK at The ANJO BBS
  • From Digital Man to MCMLXXIX on Tuesday, April 29, 2008 18:53:48
    Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Mon Apr 28 2008 09:37 pm

    Ok.. now that I've cleared up the issues surround my ability to even ask questions in here...

    I've come really far with some of these javascript games... and the next logical step is making them multiplayer (real-time)

    With the amount of data I intend to be passing back and forth between
    nodes, file IO seems like a huge PITA. I could be wrong, but that's just
    how it seems. I've had moderate success using named queues to communicate between nodes in a makeshift "chat lobby" I made... but it leaves open Queues behind in memory, which interferes with the ability to detect
    whether or not there are any other users connected.

    Named queues are *supposed* to be automatically deleted when the last "user" thread deletes their instance of the queue object (which *should* happen whenever the script terminates).

    that leaves me with sockets, unless there's a way to free up queues from memory so they don't show up in the named_queue list.

    Don't abandon the Queue idea so quickly. I'll see if I can reproduce what you're describing and fix it.

    that being said.. is there a simple way to create a local socket connection between 2 nodes? the format is a little confusing, and I've had no success so far making it work.

    File I/O or a Queue would be easier. I'd like to see if we can get the named queues working for ya.

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #56:
    1/4 of the bones in your body are in your feet.
    Norco, CA WX: 69.4°F, 56% humidity, 2 mph WSW wind, 0.00 inches rain/24hrs
  • From MCMLXXIX@MDJ to Digital Man on Wednesday, April 30, 2008 08:18:38
    Named queues are *supposed* to be automatically deleted when the last "user" thread deletes their instance of the queue object (which *should* happen whenever the script terminates).


    I first ran the multiplayer engine, then quit. It happens even if only one person runs it, but I set each script to ignore their own connection in the list so it's unnoticable.

    Thenmade a test script that loops through the list_named_queues() array and logs the name, read_level, and write_level. Sometimes they disappear, sometimes they don't. Don't know if that's of any help to you.

    ---
    ■ Synchronet ■ The BRoKEN BuBBLE (MDJ.ATH.CX)
  • From Tracker1@TRN to MCMLXXIX on Thursday, May 01, 2008 15:05:53
    On 04/29/2008 05:39 AM, MCMLXXIX wrote:
    if(!s.bind(5000, "127.0.0."+bbs.node_num)) {

    TCP gets trickier. :-)

    Ok that makes sense. I'll see what I can do with it. Thanks!

    Another suggestion if you can't use 127.0.0.* .. you could just do a
    5000 + bbs.node_num for the listening port, and have each node have a different port.. say from 5001 - 500x ...

    Kinda kludgy, but could work... not sure how something like a trivia chat would work... I would just create a service in js that each node connects to..

    --
    Michael J. Ryan - tracker1(at)theroughnecks(dot)net - www.theroughnecks.net icq: 4935386 - AIM/AOL: azTracker1 - Y!: azTracker1 - MSN/Win: (email)

    ... FRA #102: Nature decays, but latinum lasts forever.

    ---
    ■ Synchronet ■ theroughnecks.net - you know you want it
  • From MCMLXXIX@MDJ to Digital Man on Saturday, May 03, 2008 12:35:00
    Re: Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Wed Apr 30 2008 08:18:38

    Named queues are *supposed* to be automatically deleted when the last "user" thread deletes their instance of the queue object (which *should* happen whenever the script terminates).


    I first ran the multiplayer engine, then quit. It happens even if only one person runs it, but I set each script to ignore their own connection in the list so it's unnoticable.

    Thenmade a test script that loops through the list_named_queues() array and logs the name, read_level, and write_level. Sometimes they disappear, sometimes they don't. Don't know if that's of any help to you.

    Any luck duplicating those results? I'm messing around with UDP sockets, currently, but they make it kind of impossible to tell if anyone else is connected. with named queues you can at least check for their presence.. I'm really trying to avoid file i/o

    ---
    ■ Synchronet ■ The BRoKEN BuBBLE (MDJ.ATH.CX)
  • From Digital Man to MCMLXXIX on Wednesday, May 07, 2008 13:08:32
    Re: Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Sat May 03 2008 12:35 pm

    Re: Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Wed Apr 30 2008 08:18:38

    Named queues are *supposed* to be automatically deleted when the last "user" thread deletes their instance of the queue object (which *should* happen whenever the script terminates).


    I first ran the multiplayer engine, then quit. It happens even if only one person runs it, but I set each script to ignore their own connection in the list so it's unnoticable.

    Thenmade a test script that loops through the list_named_queues() array and logs the name, read_level, and write_level. Sometimes they
    disappear, sometimes they don't. Don't know if that's of any help to
    you.

    Any luck duplicating those results?

    No, I haven't tried yet. It's on my next-to-do-list, so I'll let you know soon what I find out.

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #114:
    The oldest known animal was a tortoise, which lived to be 152 years old. Norco, CA WX: 61.3°F, 68% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs
  • From MCMLXXIX@MDJ to Digital Man on Wednesday, May 07, 2008 16:45:46
    Re: Re: Queues vs. Sockets
    By: Digital Man to MCMLXXIX on Wed May 07 2008 13:08:32


    No, I haven't tried yet. It's on my next-to-do-list, so I'll let you know so what I find out.


    Ok cool. I got a form of it working nicely, using file IO and UDP sockets, but in order to transmit things like screen coordinates or anything that needs to come through and be received as it was intended, named queues seem like the best option.

    ---
    ■ Synchronet ■ The BRoKEN BuBBLE (MDJ.ATH.CX)
  • From Digital Man to MCMLXXIX on Thursday, June 12, 2008 01:11:27
    Re: Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Sat May 03 2008 12:35 pm

    Re: Re: Queues vs. Sockets
    By: MCMLXXIX to Digital Man on Wed Apr 30 2008 08:18:38

    Named queues are *supposed* to be automatically deleted when the last "user" thread deletes their instance of the queue object (which *should* happen whenever the script terminates).


    I first ran the multiplayer engine, then quit. It happens even if only one person runs it, but I set each script to ignore their own connection in the list so it's unnoticable.

    Thenmade a test script that loops through the list_named_queues() array and logs the name, read_level, and write_level. Sometimes they
    disappear, sometimes they don't. Don't know if that's of any help to
    you.

    Any luck duplicating those results?

    I tried to reproduce your problem, but could not. I created a simple script (queue.js):

    print("before:");
    print(list_named_queues());
    print("-");

    new Queue("test");
    console.pause();

    print("after:");
    print(list_named_queues());
    print("-");

    And I ran this one node, left it at the pause prompt, logged into another node and ran it again leaving the second instance also at the pause prompt. Then I hit a key on the first instance, hit a key on the second (so both scripts have now terminated) and when running it again, it shows there are no named queues. I tried letting the second script exit first and I get the same result.

    Perhaps its the way in which you're executing the script (in the same scope as your command shell maybe?). I just used ";EXEC ?queue.js", which should simulate how it would execute as a door or loadable module. Or maybe you have to write to the queue before the anomoly occurs? I don't enough information currently to know. More details on how to reproduce the problem are requested! :-)

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #116:
    The largest fish is the whale shark - it can be over 50 feet long and weigh 2 tons.
    Norco, CA WX: 64.2°F, 67% humidity, 0 mph SE wind, 0.00 inches rain/24hrs
  • From X V Lxxix to Digital Man on Thursday, June 19, 2008 08:21:05
    I tried to reproduce your problem, but could not. I created a simple script (queue.js):

    print("before:");
    print(list_named_queues());
    print("-");

    new Queue("test");
    console.pause();

    print("after:");
    print(list_named_queues());
    print("-");


    well. I'm pretty sure the problem only occurs when there is data waiting in the queue read or write buffers. when the queue is unused, it deletes as normal.

    try putting this in there:
    q=new Queue("test" + bbs.node_number);
    q.write("testing");

    that way each instance will create its own uniquely named queue and write data to it, then exit.
  • From X V Lxxix to X V Lxxix on Thursday, June 19, 2008 08:30:48
    Using the following code, I was able to duplicate my previous results..
    Once the queues are stuck, they're stuck there until you restart the bbs, I think.

    print("before:");
    print(list_named_queues());
    print("-");

    q=new Queue("test" + bbs.node_num);
    q.write("testing");
    console.pause();

    print("after:");
    print(list_named_queues());
    print("-");
  • From Digital Man to X V Lxxix on Thursday, June 19, 2008 11:12:53
    Re: Re: Queues vs. Sockets
    By: X V Lxxix to X V Lxxix on Thu Jun 19 2008 08:30 am

    Using the following code, I was able to duplicate my previous results..
    Once the queues are stuck, they're stuck there until you restart the bbs, I think.

    print("before:");
    print(list_named_queues());
    print("-");

    q=new Queue("test" + bbs.node_num);
    q.write("testing");
    console.pause();

    print("after:");
    print(list_named_queues());
    print("-");

    Try declaring 'q' as a local variable:

    var q=new Queue("test" + bbs.node_num);

    And/Or specifically deleting the Queue instance at the end of the script:

    delete q;

    Both (either one) of these fixes forces the Queue destructor to be called at the end of the script which is responsible for detaching the node thread from the Queue and removing the Queue from the named-queue list.

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #76:
    The average person spends 2 years on the phone in his/her lifetime.
    Norco, CA WX: 64.2°F, 67% humidity, 0 mph SE wind, 0.00 inches rain/24hrs
  • From X V Lxxix to Digital Man on Thursday, June 19, 2008 12:46:57
    Try declaring 'q' as a local variable:

    var q=new Queue("test" + bbs.node_num);

    And/Or specifically deleting the Queue instance at the end of the script:

    delete q;

    Both (either one) of these fixes forces the Queue destructor to be called at the end of the script which is responsible for detaching the node thread from the Queue and removing the Queue from the named-queue list.

    that worked. I think part of the problem was that to make the queue communications work for more than 2 people, I had to create more than one queue, and consequently each participating user had to create a local instance of each named queue.

    It made things a lot harder to clean up. Since you have a local instance of every user's queue, you can't rely on the queue's existence to determine if the user is still present. so I had to make an exiting user broadcast on a named "quit" channel so that listening connections would know the person has left, and both would have to delete their instance of the queue.

    Somewhere in that nonsense things did not work.
  • From Digital Man to X V Lxxix on Thursday, June 19, 2008 13:02:50
    Re: Re: Queues vs. Sockets
    By: X V Lxxix to Digital Man on Thu Jun 19 2008 12:46 pm

    Both (either one) of these fixes forces the Queue destructor to be
    called at the end of the script which is responsible for detaching the node thread from the Queue and removing the Queue from the named-queue list.

    that worked. I think part of the problem was that to make the queue communications work for more than 2 people, I had to create more than one queue, and consequently each participating user had to create a local instance of each named queue.

    It made things a lot harder to clean up. Since you have a local instance of every user's queue, you can't rely on the queue's existence to determine if the user is still present. so I had to make an exiting user broadcast on a named "quit" channel so that listening connections would know the person
    has left, and both would have to delete their instance of the queue.

    Somewhere in that nonsense things did not work.

    Glad it's working for you now. In general, it's good practice to use 'var' to define variables as the scope is then specified (not global) and you'll have better garbage collection.

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #127:
    A hummingbird's heart beats 1,400 times a minute.
    Norco, CA WX: 64.2°F, 67% humidity, 0 mph SE wind, 0.00 inches rain/24hrs
  • From Digital Man to X V Lxxix on Thursday, June 19, 2008 17:37:19
    Re: Re: Queues vs. Sockets
    By: Digital Man to X V Lxxix on Thu Jun 19 2008 01:02 pm

    Re: Re: Queues vs. Sockets
    By: X V Lxxix to Digital Man on Thu Jun 19 2008 12:46 pm

    Both (either one) of these fixes forces the Queue destructor to be called at the end of the script which is responsible for detaching
    the node thread from the Queue and removing the Queue from the named-queue list.

    that worked. I think part of the problem was that to make the queue communications work for more than 2 people, I had to create more than
    one queue, and consequently each participating user had to create a
    local instance of each named queue.

    It made things a lot harder to clean up. Since you have a local instance of every user's queue, you can't rely on the queue's existence to determine if the user is still present. so I had to make an exiting user broadcast on a named "quit" channel so that listening connections would know the person has left, and both would have to delete their instance
    of the queue.

    Somewhere in that nonsense things did not work.

    Glad it's working for you now. In general, it's good practice to use 'var' to define variables as the scope is then specified (not global) and you'll have better garbage collection.

    BTW, I realize it's probably not clear from the documentation (http://synchro.net/docs/jsobjs.html#Queue), but the "value" argument to the Queue.write() method can be any of several JS value types: bool, string, number, null, *even* an object (with properties of any of these types, including nested objects).

    This may help in passing complex state information between threads (nodes) without requiring complex string parsing.

    digital man (xbox-live: digitlman)

    Snapple "Real Fact" #76:
    The average person spends 2 years on the phone in his/her lifetime.
    Norco, CA WX: 64.2°F, 67% humidity, 0 mph SE wind, 0.00 inches rain/24hrs
  • From X V Lxxix to Digital Man on Friday, June 20, 2008 07:34:25
    Re: Re: Queues vs. Sockets
    By: Digital Man to X V Lxxix on Thu Jun 19 2008 01:02 pm

    Re: Re: Queues vs. Sockets
    By: X V Lxxix to Digital Man on Thu Jun 19 2008 12:46 pm

    Both (either one) of these fixes forces the Queue destructor to be called at the end of the script which is responsible for detaching the node thread from the Queue and removing the Queue from the named-queue list.

    that worked. I think part of the problem was that to make the queue communications work for more than 2 people, I had to create more than one queue, and consequently each participating user had to create a local instance of each named queue.

    It made things a lot harder to clean up. Since you have a local instance of every user's queue, you can't rely on the queue's existence to determine if the user is still present. so I had to make an exiting user broadcast on a named "quit" channel so that listening connections would know the person has left, and both would have to delete their instance of the queue.

    Somewhere in that nonsense things did not work.

    Glad it's working for you now. In general, it's good practice to use 'var' to define variables as the scope is then specified (not global) and you'll have better garbage collection.

    BTW, I realize it's probably not clear from the documentation (http://synchro.net/docs/jsobjs.html#Queue), but the "value" argument to the Queue.write() method can be any of several JS value types: bool, string, number, null, *even* an object (with properties of any of these types, including nested objects).

    This may help in passing complex state information between threads (nodes) without requiring complex string parsing.


    VERY good to know. For some reason I was under the assumption that was not the case. I think I tried passing an object and it didn't work. But....... I've been known to pull on the "push to open" doors.