• bug in binkd flo file processing?

    From mark lewis@1:3634/12.73 to all on Saturday, June 22, 2019 20:36:50

    i have a mail tosser that creates flo files containing "../" at the beginning of the line pointing to the files to be sent... this is fine in most cases because we generally change directory to a proper place before starting the mailer...

    yesterday we had a power outage... when the power was restored and i was bring the system back online i forgot to change to the proper directory before starting binkd... as a result, it could not find the files it was trying to send and emitted messages like this...

    start_file_transfer: ../fido/outbound/5d02ac39.pkt: No such file or directory

    ok, i don't have a problem with that... the problem and the reason for this post is that the flo file is deleted thus leaving behind a file with no indication as to its destination...


    my query is if binkd should be deleting the flo file when it cannot file the file(s) to send... i would think that if there is an error like this, binkd would elect to be cautious and not delete the flo file or to at least recreate it as it was like it does in other instances (eg: connection failure)...


    i was able to recover and get the several thousand pkts sent but this is not a fun task to deal with...

    yes, i'll also be checking with the mail tosser developer and seeing if we can get the full path in place instead of using the "../blah/blah" format...

    thank you for your time and attention :)

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... System Update - A quick method of trashing ALL of your software.
    ---
    * Origin: (1:3634/12.73)
  • From Tommi Koivula@2:221/360 to mark lewis on Sunday, June 23, 2019 09:55:46

    On 23/06/2019 3.36 you wrote:

    yes, i'll also be checking with the mail tosser developer and seeing if
    we can get the full path in place instead of using the "../blah/blah"
    format...

    Meanwhile, how about using ftrans in binkd config to map ../ to absolute path?

    ---
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/360)
  • From mark lewis@1:3634/12.73 to Tommi Koivula on Sunday, June 23, 2019 08:55:02

    On 2019 Jun 23 09:55:46, you wrote to me:

    yes, i'll also be checking with the mail tosser developer and seeing if
    we can get the full path in place instead of using the "../blah/blah"
    format...

    Meanwhile, how about using ftrans in binkd config to map ../ to
    absolute path?

    oh! never thought about that... so something like

    ftrans "../" "/sbbs/"

    should be all it takes?? does it take regex? i think i'd like to protect it with something like

    ftrans "^../" "/sbbs/"

    so that it only affects leading "../" and not one that may appear in the middle
    of the path...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Steer clear of incorrect forms of verbs that have snuck in the language. ---
    * Origin: (1:3634/12.73)
  • From Michael Dukelsky@2:5020/1042 to mark lewis on Sunday, June 23, 2019 19:08:02
    Hello mark,

    Sunday June 23 2019, mark lewis wrote to Tommi Koivula:

    Meanwhile, how about using ftrans in binkd config to map ../ to
    absolute path?

    oh! never thought about that... so something like

    ftrans "../" "/sbbs/"

    should be all it takes?? does it take regex?

    No, neither regular expressions nor wildcards are supported in ftrans. It takes
    two strings as they are.

    Michael

    ... node (at) f1042 (dot) ru
    --- GoldED+/LNX 1.1.5-b20170303
    * Origin: Moscow, Russia (2:5020/1042)
  • From Michiel van der Vlist@2:280/5555 to Michael Dukelsky on Sunday, June 23, 2019 19:34:11
    Hello Michael,

    On Sunday June 23 2019 19:08, you wrote to mark lewis:

    ftrans "../" "/sbbs/"

    should be all it takes?? does it take regex?

    No, neither regular expressions nor wildcards are supported in ftrans.
    It takes two strings as they are.

    If one needs a moe pwerfull serial editor there is sed. It processes regular expressions and wildcards. There are versions for Linux and Windows.


    Cheers, Michiel

    --- GoldED+/W32-MSVC 1.1.5-b20170303
    * Origin: http://www.vlist.eu (2:280/5555)
  • From Tommi Koivula@2:221/360 to Michiel van der Vlist on Sunday, June 23, 2019 20:47:53
    On 23.06.2019 20:34, Michiel van der Vlist wrote:

    Hello Michael,

    On Sunday June 23 2019 19:08, you wrote to mark lewis:

     ml>>   ftrans "../" "/sbbs/"

     ml>> should be all it takes?? does it take regex?

     MD> No, neither regular expressions nor wildcards are supported in ftrans.  MD> It takes two strings as they are.

    If one needs a moe pwerfull serial editor there is sed. It processes
    regular expressions and wildcards. There are versions for Linux and
    Windows.

    Say What?!-)

    Please do read this thread from the beginning, then search binkd docs
    for "ftrans", and then reprase your comment. :D

    'Tommi

    ---
    * Origin: nntp://rbb.fidonet.fi - Lake Ylo - Finland (2:221/360)
  • From Jeff Smith@1:282/1031 to Mark Lewis on Sunday, June 23, 2019 18:52:30
    Hello Mark,

    i was able to recover and get the several thousand pkts sent but this is not a
    fun task to deal with...

    More of a task than I would care to do for sure.

    yes, i'll also be checking with the mail tosser developer and seeing if we can
    get the full path in place instead of using the "../blah/blah" format...

    I have run into that situation also. I tend to specify the full path's Linux path in my configs to hopefully avoid any doubt.

    Jeff


    --- BBBS/Li6 v4.10 Toy-4
    * Origin: Fidoneet: The Ouija Board - Anoka, MN -bbs.ouijabrd.net (1:282/1031)
  • From mark lewis@1:3634/12.73 to Jeff Smith on Wednesday, June 26, 2019 04:32:10

    On 2019 Jun 23 18:52:30, you wrote to me:

    i was able to recover and get the several thousand pkts sent but this
    is not a fun task to deal with...

    More of a task than I would care to do for sure.

    thankfully there was an easy to use binary tool to analyze the raw pkts and emit their source and destination addresses... it sure beat having to look at the pkts manually for their passwords to figure them out... those without passwords certainly needed to be analyzed for their destination addresses... once we had the destination addresses, it was a simple matter to recreate the flo files...

    eg:
    # make a list of pkt destination addresses
    /sbbs/exec/pktdump /sbbs/fido/outbound/*.pkt | \
    egrep -e "Packet Type .* from $MYFTNADDR to " | \
    cut -d " " -f 12 | sort -V | uniq

    # choose an address from the above list
    # get a list of pkts destined there and
    # move those pkts to the tmp directory
    /sbbs/exec/pktdump /sbbs/fido/outbound/*.pkt | \
    egrep -e "Packet Type .* from $MYFTNADDR to $DESTADDR" | \
    cut -d " " -f 1 | tr '\n' '\0' | \
    xargs -0 -n1 -i mv '{}' /sbbs/tmp

    # create a flo file with the pkts
    printf "$(ls -d -1 $PWD/*.pkt | \
    sed 's/^/\^/')\n" > \
    $(printf "%04x%04x.flo\n" $DESTNET $DESTNODE)

    it can be done a bit better by not moving the pkts to the tmp directory first but this works fine with some manual work added... once the flo file is created, it is a simple matter manually to copy/move it to the proper outbound directory and wait for the mailer to transit and delete the pkts listed in it before handling the next system in the list of destinations...

    i've not yet spent more time to fix it up into a proper script that doesn't need to move the files at all plus there should be some .bsy file creation to prevent the tosser and mailer from jumping the gun while the pkts are moved and
    the flo is created... i've already got script code that breaks down FTN addresses into their component parts and determines the proper outbound directory so that part is already done... i just need to find that source library and use it for this, too...

    yes, i'll also be checking with the mail tosser developer and seeing if
    we can get the full path in place instead of using the "../blah/blah"
    format...

    I have run into that situation also. I tend to specify the full path's Linux path in my configs to hopefully avoid any doubt.

    yeah, unfortunately i don't think the operator defined paths in the configs have much to do with the software's output of relative directory usage... i understand the reasoning for using relative directories but it can be problematic when mixing in 3rd party software... i do need to visit the wiki, though, and see if i can find something there that might point to a setting that might cover this...

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... We must all band together and be non-conformists.
    ---
    * Origin: (1:3634/12.73)