Hi All,

I'm running the 386 MS-DOS flavor of Squish.

How large of *.pkt files can it handle, newgroups files, sometimes HUGE files.


Kevin
klement@gypsy-designs.com

--- Squish/386 v1.11
* Origin: Gypsy BBS -- Gypsy Designs CDN (403) 242-3221 (1:342/77)

HI KEvin,

On Wed 2038-Mar-17 18:28, Kevin Klement (1:342/77) wrote to All:

I'm running the 386 MS-DOS flavor of Squish.

How large of *.pkt files can it handle, newgroups files, sometimes
HUGE files.

I've never run into a problem. IN fact, I've never heard,
or read anything about a limitation in packet size. Older
vers of squish, iirc had a 16k message size limit. I"m
running the dos 386 ver as you are, and mine seems to split
overly long messages <knock on pressboard> whereas it's said that older version
would rename a pkt with a long message to
*.lng and not toss the messages, but I"ve never seen that
behavior.

But, in case some folks are running mail processors that
barf on anything bigger than 16k if I post longer form
pieces such as some postings I make to the ls_arrl echo of
ham radio news items I split the input text file at 16k. I
figre that should handle most of them, and I've seen no
complaints.


Regards,

Richard
--- timEd 1.10.y2k+
* Origin: Radio REscue net operations BBS (1:116/901)

Hello Kevin!

17 Mar 10 18:28, Kevin Klement wrote to All:

I'm running the 386 MS-DOS flavor of Squish.

How large of *.pkt files can it handle, newgroups files, sometimes
HUGE files.

It's a shame if your newsgroup util doesn't have enough sense to build more than one PKT, but I don't think the PKT size will matter to Squish as long as you have the disk space for the PKT and MSG versions of the data.

What =will= matter is the size of the individual messages, as they will get stored in RAM after they are extracted from the PKTs but before they are inserted into the messagebase. That will depend on your "Buffers" statement in
the config:

===
The defaults for the 32-bit versions of Squish (SQ386 and
SQ386P) are:

BUFFERS writebuf outbuf msgbuf
Small 64k 64k 64k
Medium 76k 128k 128k
Large 128k 512k 256k
===

You can also assign your own specific values for each buffer if you don't like the Small/Medium/Large shortcuts.

.\\ike

--- GoldED 2.50+
* Origin: -=( The TechnoDrome )=- Austin,TX 512-327-8598 33.6k (1:382/61)

Hi Mike,

On Thu 2038-Mar-18 00:33, Mike Tripp (1:382/61) wrote to Kevin Klement:

Hello Kevin!
It's a shame if your newsgroup util doesn't have enough sense to
build more than one PKT, but I don't think the PKT size will matter
to Squish as long as you have the disk space for the PKT and MSG
versions of the data.

Agreed. I think a feed provides me some newsgroups uses
gigo, and I'm having no troubles processing them here.

What =will= matter is the size of the individual messages, as they
will get stored in RAM after they are extracted from the PKTs but
before they are inserted into the messagebase. That will depend on
your "Buffers" statement in the config:

Agreed again. I think, the behavior I mentioned earlier
this thread of splitting large messages works because I"m
using the large buffers setting. OTherwise ymmv. I think
the docs mention something about the behavior changing
according to buffer size, i.e. squish will behave as it did
with earlier versions if you use medium or small buffers.

Regards,
Richard
--- timEd 1.10.y2k+
* Origin: Radio REscue net operations BBS (1:116/901)

Hello Richard!

18 Mar 10 15:23, Richard Webb wrote to Mike Tripp:

Agreed again. I think, the behavior I mentioned earlier
this thread of splitting large messages works because I"m
using the large buffers setting. OTherwise ymmv. I think
the docs mention something about the behavior changing
according to buffer size, i.e. squish will behave as it did
with earlier versions if you use medium or small buffers.

Squish does not do any message splitting...though I do not recall whether it renames the PKT to LNG or BAD for later reprocessing. The only change I'm aware of between versions was new numeric values for Small/Medium/Large and the
ability to assign them (or explicit numeric values) to the 3 individual buffers.

I did all sorts of convolutions to get a utility called PKTSort to do the splitting by pre-processing the inbound directory while it was actually designed to post-process the outbound directory. I eventually found a newsgroup
gating util called NGate which bypasses tossing and transfers messages directly
between NNTP server and squishbase...and still did a batch manipulation of the Buffers statement in SQUISH.CFG just for scanning the newsgroups out to downlinks.

After monitoring the messages >256K for a bit, I decided that I didn't really need to be forwarding the spam or uuencoded "stuff" that either belonged in a binary group or the bit-bucket anyway.

ymmv also,
.\\ike

--- GoldED 2.50+
* Origin: -=( The TechnoDrome )=- Austin,TX 512-327-8598 33.6k (1:382/61)

HI Mike,

On Fri 2038-Mar-19 12:08, Mike Tripp (1:382/61) wrote to Richard Webb:

this thread of splitting large messages works because I"m
using the large buffers setting. OTherwise ymmv. I think
the docs mention something about the behavior changing
according to buffer size, i.e. squish will behave as it did
with earlier versions if you use medium or small buffers.

Squish does not do any message splitting...though I do not recall
whether it renames the PKT to LNG or BAD for later reprocessing.
The only change I'm aware of between versions was new numeric values
for Small/Medium/Large and the ability to assign them (or explicit
numeric values) to the 3 individual buffers.

Okay, they might be split upstream of me, been a long time.
I know stuff originated here is split before I even let
squish see it if I think it's going to be overlong.

I did all sorts of convolutions to get a utility called PKTSort to
do the splitting by pre-processing the inbound directory while it
was actually designed to post-process the outbound directory. I
eventually found a newsgroup gating util called NGate which bypasses tossing and transfers messages directly between NNTP server and squishbase...and still did a batch manipulation of the Buffers
statement in SQUISH.CFG just for scanning the newsgroups out to
downlinks.

YEp, I got pktsort to be happy by using gus to break open
the archive bundles first then ran pktsort across the
incoming packets back in my busy hub days. That was my
workaround anyway.
Worked well, just invoke gus after receiving compressed mail on the appropriate
inbound with the /m command line switch,
then run pktsort.

Regards,
Richard
--- timEd 1.10.y2k+
* Origin: Radio REscue net operations BBS (1:116/901)

Hello Richard!

18 Mar 10 15:23, Richard Webb wrote to Mike Tripp:

Agreed again. I think, the behavior I mentioned earlier
this thread of splitting large messages works because I"m
using the large buffers setting. OTherwise ymmv. I think
the docs mention something about the behavior changing
according to buffer size, i.e. squish will behave as it did
with earlier versions if you use medium or small buffers.

Squish does not do any message splitting...though I do not recall whether it renames the PKT to LNG or BAD for later reprocessing. The only change I'm aware of between versions was new numeric values for Small/Medium/Large and the
ability to assign explicit numeric values to each of the 3 individual buffers.

PKTSort was a popular tool for message-splitting, but it was designed to be used to post-process your outbound to split large messages after they were already scanned. I went through a bunch of convoluted tricks to get it to pre-process inbound created by the newsgating util. I was eventually able to eliminate the intermediate format. pre-processing, and tossing altogether using
NGate to transfer between NNTP server and Squishbases directly. I still a bit of batch work to toggle the Squish buffers from Small to Large, scan the newsgroup areas to downlinks, and then toggle from Large back to Small for normal processing again.

After monitoring the messages >256K for a bit, I decided that I didn't really need to be forwarding the spam or uuencoded "stuff" that either belonged in a binary group or trash that belonged in the bit-bucket anyway.

ymmv also,
.\\ike

--- GoldED 2.50+
* Origin: -=( The TechnoDrome )=- Austin,TX 512-327-8598 33.6k (1:382/61)

Hi Richard,

Thursday March 18 2010 02:31, Richard Webb wrote to Kevin Klement:

I've never run into a problem. IN fact, I've never heard,
or read anything about a limitation in packet size.

'k

vers of squish, iirc had a 16k message size limit. I"m
running the dos 386 ver as you are, and mine seems to split

Soupgate, the MINE converter I use splites long msg'a anyways.

Thanks hu.


Kevin
klement@gypsy-designs.com

--- Squish/386 v1.11
* Origin: Gypsy BBS -- Gypsy Designs CDN (403) 242-3221 (1:342/77)

Hi Mike,

Thursday March 18 2010 00:33, Mike Tripp wrote to Kevin Klement:

It's a shame if your newsgroup util doesn't have enough
sense to build more than one PKT, but I don't think the PKT

Actually, Vsoup the User-Agent I use DL's MINE files, and it's configurable, as many msg per file. I DL'ed 10mg MINE file once, stupid ideas, there was over 3000+ *.pkt file to toss.

That will depend on your "Buffers" statement in the config:

Got it.

Thanks eh.

Kevin
klement@gypsy-designs.com

--- Squish/386 v1.11
* Origin: Gypsy BBS -- Gypsy Designs CDN (403) 242-3221 (1:342/77)

Hi Richard,

Friday March 19 2010 22:05, Richard Webb wrote to Mike Tripp:

YEp, I got pktsort to be happy by using gus to break open
the archive bundles first then ran pktsort across the
incoming packets back in my busy hub days. That was my
workaround anyway.

Me too, when I was HUBing newsgroups I used pksort also, my uplink always sent HUGE *.pkt files, Soupgate is configurable in this area, so I don't need pksort now.


Kevin
klement@gypsy-designs.com

--- Squish/386 v1.11
* Origin: Gypsy BBS -- Gypsy Designs CDN (403) 242-3221 (1:342/77)