Moin Eddy!

10.03.2001, Eddy Thilleman wrote a message to Vitus Jensen:

I've written a sweep command in pascal and compiled it with
Borland Pascal as DOS .EXE file and compiled it with Virtual
Pascal/2 v2.1 build 243 as OS/2 .EXE file, and implemented it
also in REXX.

...

another example

sweep for %a in (*.bak) do echo %a

The compiled version is easier to use than the REXX variant.

"easier to use"? I would expect the rexx variant to be easier to code.


...

Shall I post the source of all three?

Are you sure it's the screen output?

No, I'm not certain.

If not, remove screen output from the program and rerun it.

I've run it without screen output from all the sweep variants
themselves, the only screen output is by the command run in each (sub)directory. The command in the compiled OS/2 version is run
somewhat slower.

There are larger gaps between execution in the different directories? That would mean that searching for directories is slower in VP/2 (how do you do it?), which is not to be expected or the startup of every command line takes longer (s.b.).

If you know the offending part in the OS/2 EXE it may be an idea to
post that part.

I don't know what the cause is. My best guess is that the loading and initializing of cmd.exe takes more time than command.com. Cmd.exe and command.com are located on the same partition. In a few days (I don't
have the time at the moment), I'll put a copy of cmd.exe on a ramdisk
and modify the sweep pascal source code to start the cmd.exe file on
the ramdisk, to see if that cuts down the difference.

cmd.exe should already reside in the disk cache so the difference should be small or non existent.

You are executing all commands through cmd.exe when coding in pascal. How do you do it in Rexx? If you just let the already running copy of cmd.exe handle it we have a major difference: a new process for cmd.exe does not have to be started.
Try to compare the following times,

Rexx: sweep cmd /C "for %a in (*.bak) do echo %a"
Pascal: sweep for %a in (*.bak) do echo %a


Bye,
Vitus

---
* Origin: REALITY.SYS Corrupted: Re-boot universe? (Y/N/Q) (2:2474/424.1)

Hello Vitus,

Wednesday 14 March 2001 02:18, Vitus Jensen wrote to Eddy Thilleman:

Borland Pascal as DOS .EXE file and compiled it with Virtual
Pascal/2 v2.1 build 243 as OS/2 .EXE file, and implemented it
also in REXX.

...

another example

sweep for %a in (*.bak) do echo %a

The compiled version is easier to use than the REXX variant.

"easier to use"? I would expect the rexx variant to be easier to
code.

Yep, that's the difference. The REXX variant is somewhat easier to code (the pascal version is not difficult), but I found it more difficult to use the REXX
variant with a "for %a in (*) do..." command (with and without a preceding 'cmd
/c'), because sometimes I get errors, like:

\os2\rexx\sweep for %a in (*.txt) do echo %a

Command >> for %a in (*.txt) do echo %a

Sweep >>> C:\

SYS1079: a was unexpected at this time.
Error? rc=1

\os2\rexx\sweep cmd /c for %a in (*.txt) do echo %a

Command >> cmd /c for %a in (*.txt) do echo %a

Sweep >>> C:\

SYS1079: a was unexpected at this time.
Error? rc=1

\os2\rexx\sweep cmd /c for %%a in (*.txt) do echo %%a

Command >> cmd /c for %%a in (*.txt) do echo %%a

Sweep >>> C:\

Error? rc=31112

\os2\rexx\sweep cmd /c for %a in (*.txt) do echo %a

Command >> cmd /c for %a in (*.txt) do echo %a

Sweep >>> C:\

SYS1079: a was unexpected at this time.
Error? rc=1


Only one time I managed to get above construct working with the REXX variant. That's why I wrote the pascal version.

Shall I post the source of all three?

Are you sure it's the screen output?

No, I'm not certain.

If not, remove screen output from the program and rerun it.

I've run it without screen output from all the sweep variants
themselves, the only screen output is by the command run in each
(sub)directory. The command in the compiled OS/2 version is run
somewhat slower.

There are larger gaps between execution in the different directories?

Yes.

That would mean that searching for directories is slower in VP/2
which is not to be expected or the startup of every command line takes longer (s.b.).

I'm not sure about this.

(how do you do it?),

FindFirst/FindNext.

Here is the source code of the pascal version:

------------------- begin of sweep.pas -------------------

uses dos;

Var
i : byte;
quiet : boolean;
CmdProc : string;
CmdLine : string;


function RunProg( Prog, Parm: string ): boolean;
begin
Exec( Prog, Parm );
if DOSError = 0 then
RunProg := True
else
writeln( 'DOS error #', DOSError );
end;


{$S+}
Procedure AllDirs;
var
DR : SearchRec;
Pad: string;

begin
GetDir( 0, Pad );
if not quiet then writeln( '>>> SWEEP >>> ' + Pad );
if RunProg( CmdProc, '/c' + CmdLine ) then
begin
FindFirst('*', Directory, DR);
while (DosError = 0) do
begin
if (DR.Attr and Directory) = Directory then
begin
if ((DR.Name <> '.') and (DR.Name <> '..')) then
begin
ChDir( DR.Name );
AllDirs;
ChDir( '..' );
end
end;
FindNext( DR );
end;
end;
end;
{$S-}

begin
quiet := False;
if ParamCount > 0 then begin
CmdProc := GetEnv('COMSPEC');
CmdLine := '';
for i := 1 to ParamCount do
begin
if ((paramstr( i )[1] = '/') or (paramstr( i )[1] = '-')) and (upcase(paramstr( i )[2]) = 'Q') then
quiet := True
else
CmdLine := CmdLine + ' ' + paramstr( i );
{ /q ? }
end; { all parameters }
writeln( 'command:', CmdLine );
AllDirs;
end;
end.
-------------------- end of sweep.pas --------------------

cmd.exe should already reside in the disk cache so the difference
should be small or non existent.

You are executing all commands through cmd.exe when coding in pascal.
How do you do it in Rexx?

Here is the source code for the REXX variant:

------------------- begin of sweep.cmd -------------------
/*
* A sweep command for OS/2
* Recurses through directories from default and executes a command line
*/

'@echo off'

ARG P1 dummy

if P1 = '-Q' then
parse arg dummy cl
else do
P1 = 0
parse arg cl
end

if cl = '/?' then signal GiveHelp
if cl = '?' then signal GiveHelp
if cl = '-?' then signal GiveHelp
if cl = '' then signal GiveHelp

signal on Syntax
signal on Error
signal on Failure
signal on Halt

CRLF = '0D'x || '0A'x

n = setlocal()

say '>> Command >>' cl
call Exec( directory() )

n = endlocal()
return


Exec: procedure expose cl P1
parse arg path

newdir = directory( path )
if newdir = path then
do
if P1 \= '-Q' then say '>>> Sweep >>>' directory()
cl
end

Call SysFileTree path || "\*", 'dir', 'D'
if datatype( dir.0 ) <> "NUM" then dir.0 = 0
do f = 1 to dir.0
name = substr( dir.f, 38 )
if (substr( dir.f, 32, 1 ) = 'D') then /* directory? */
call Exec( name )
end
return


Syntax:
ErrorNr = rc
Say 'Line' sigl':' Sourceline(sigl)
Say 'Error' rc':' Errortext(ErrorNr)
n = endlocal()
exit


Failure:
say 'Failure, rc='rc
n = endlocal()
exit


Error:
say 'Error? rc='rc
n = endlocal()
exit


Halt:
n = endlocal()
exit


GiveHelp:
say 'SWEEP.CMD'
say ' Performs a command in the current directory and in all its subdirectories.'
say ''
say ' Usage: Sweep [-Q] command'
say ''
say ' -Q quiet, don''t display directories'
exit
-------------------- end of sweep.cmd --------------------

If you just let the already running copy of cmd.exe handle it we have
a major difference: a new process for cmd.exe does not have to be
started. Try to compare the following times,

The DOS pascal version is equally fast as the REXX version, the OS/2 VP/2 version is slower (as I said).

Rexx: sweep cmd /C "for %a in (*.bak) do echo %a"

I can't execute this with the REXX version. single commands work with the REXX version, however.

That's why I just give cd as the single command to all sweep versions.

Pascal: sweep for %a in (*.bak) do echo %a

for the pascal version, this is no problem.

Any ideas?


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... Warp 4, Scotty... and close those damn Windows!
--- GoldED/2 3.0.1
* Origin: The only thing more complicated than computers is TAX (2:280/5143.7)

Hello Vitus,

Tuesday 06 March 2001 09:22, Vitus Jensen wrote to Eddy Thilleman:

I've written a sweep command in pascal and compiled it with
Borland Pascal as DOS .EXE file and compiled it with Virtual
Pascal/2 v2.1 build 243 as OS/2 .EXE file, and implemented it
also in REXX.

I'm still uncertain what a "sweep" program means. I program which
searches the disk for certain files (.bak, etc) and removes them?

No, sorry. Sweep is a program that runs the as parameter (on the commandline) given command in the current directory and in all of its subdirectories.

for example:

sweep dir

runs the dir command in the current + all its subdirectories


another example

sweep for %a in (*.bak) do echo %a

The compiled version is easier to use than the REXX variant.

The usual reason is that you display one character at a time. If you
have control over the code which does the screen output (e.g. you are directly calling Vio*) try to buffer characters.

I use the standard writeln Pascal statement, I don't know if that's the cause.

I haven't looked at the underlying source code. I don't know if I have the underlying source code for the writeln statement in VP/2, because I can't find the source code for VP/2's system unit.

Shall I post the source of all three?

Are you sure it's the screen output?

No, I'm not certain.

If not, remove screen output from the program and rerun it.

I've run it without screen output from all the sweep variants themselves, the only screen output is by the command run in each (sub)directory. The command in
the compiled OS/2 version is run somewhat slower.

If you know the offending part in the OS/2 EXE it may be an idea to
post that part.

I don't know what the cause is. My best guess is that the loading and initializing of cmd.exe takes more time than command.com. Cmd.exe and command.com are located on the same partition. In a few days (I don't have the time at the moment), I'll put a copy of cmd.exe on a ramdisk and modify the sweep pascal source code to start the cmd.exe file on the ramdisk, to see if that cuts down the difference.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... The NEW XT EMULATOR! MicroSoft Windows NT
--- GoldED/2 3.0.1
* Origin: OS/2 in '92! (2:280/5143.7)

Hello Eddy,

On 15/Mar/01 at 11:55 you wrote:

Borland Pascal as DOS .EXE file and compiled it with Virtual
Pascal/2 v2.1 build 243 as OS/2 .EXE file, and implemented it
also in REXX.

FindFirst('*', Directory, DR);
while (DosError = 0) do
begin
if (DR.Attr and Directory) = Directory then
begin
if ((DR.Name <> '.') and (DR.Name <> '..')) then
begin
ChDir( DR.Name );
AllDirs;
ChDir( '..' );
end
end;
FindNext( DR );
end;

Call SysFileTree path || "\*", 'dir', 'D'
if datatype( dir.0 ) <> "NUM" then dir.0 = 0
do f = 1 to dir.0
name = substr( dir.f, 38 )
if (substr( dir.f, 32, 1 ) = 'D') then /* directory? */
call Exec( name )
end

The DOS pascal version is equally fast as the REXX version, the
OS/2 VP/2 version is slower (as I said).

I think it is possible, that SysFileTree and OS/2 DOS support are simply caching directory search and do not call DosFindFirst/DosFindNext for every filename. These functions allow to request information about ~64K filenames at once, and IMHO this can result in speeding up the process greately.

While SysFileTree is obviously must use such a technique, I am in doubt if OS/2
DOS support does so actually. Try to replace running external programs with somewhat fast enough (may be without any output) to figure out if the main brake is FindFirst/FindNext or not.

Cheers,
\Lesha

--- MadMED v0.42i/OS2 (Apr 25 1997 15:17:26)
* Origin: HDD2G/HPFS+HDD32M/RT-11 (2:5020/1613)

Hello Lesha,

Monday 19 March 2001 10:55, Lesha Tsoorgaev wrote to Eddy Thilleman:

I think it is possible, that SysFileTree and OS/2 DOS support are
simply caching directory search and do not call
DosFindFirst/DosFindNext for every filename. These functions allow to request information about ~64K filenames at once, and IMHO this can
result in speeding up the process greately.

Why not for the OS/2 version, I don't know.

While SysFileTree is obviously must use such a technique, I am in
doubt if OS/2 DOS support does so actually. Try to replace running
external programs with somewhat fast enough (may be without any
output) to figure out if the main brake is FindFirst/FindNext or not.

I used just the 'cd' command to test this, that's fast enough I think. :)

I haven't had time to investigate this further in the meantime, but I'm also out of ideas about this.

I should problably investigate this further at the OS/2 API level, when I've time.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... A computer cuts your work in half and gives you back the bloody ends.
--- GoldED/2 3.0.1
* Origin: Programmers don't die, they just GOSUB without RETURN (2:280/5143.7)

While SysFileTree is obviously must use such a technique, I

> am in doubt if OS/2 DOS support does so actually. Try to
> replace running external programs with somewhat fast enough
> (may be without any output) to figure out if the main brake
> is FindFirst/FindNext or not.

The DOS support in OS/2 does appear to use a badly written
FindFirst/FindNext call, however it has a serious bug that
results in a return of a maximum of about 38,000 files per call.


LRA


-- SPEED 2.01 #2720: Nostalgia isn't what it used to be.

--- Maximus/2 3.01
* Origin: Top Hat 2 BBS (1:343/41)

uses dos;

I'm very suspicious of a library of supposed wrappers around system API functions that

(a) is called "dos", and
(b) doesn't appear to have a FindClose() function.

I'd worry that the library was structured to operate in terms of how the DOS system API operates, rather than how the OS/2 system API operates. (DOS does directory searches using a single, fixed, data buffer that doesn't need to be opened and closed. OS/2 has a handle-based system for searching directories, where the handles need to be closed after use.)

Alas! All too many people, including library implementors, think that designing things to operate in the way that DOS operates is the way to write "portable" programs, because of some shoddy reasoning on their part that DOS is
some sort of "least common denominator". It isn't. And designing things to operate in the way that DOS operates is in fact a good route to producing *non*-portable programs.

Even aside from the paradigm differences between DOS and better operating systems such as OS/2, it's the way to produce grossly inefficient programs. It
is often inefficient to implement an interface that operates in DOS terms on top of an operating system such as OS/2. Just look at the many hoops that the libraries in most C implementations for OS/2 have to jump through in order to implement compatibility functions for the C language bindings to the DOS system
API such as findfirst() and findnext(). In DOS, the C language bindings to the
DOS system API are efficient, because the mapping is direct. Implementing the DOS system API on top of the OS/2 system API is not necessarily so efficient, because the mapping is not always an identity.

Try rewriting your Pascal application to call the DosFind{First,Next,Close} functions in the OS/2 system API directly (if such is possible in Virtual Pascal) rather than using this "dos" library, whatever it is.

» JdeBP «

--- FleetStreet 1.22 NR
* Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

Exec( Prog, Parm );
if DOSError = 0 then

I'm also suspicious of a function called "Exec" that takes so few parameters. What goes on inside this function ? Do you know ? Does it call DosExecPgm() after massaging "Prog" and "Parm" into appropriate form ? Does it run the program indicated by "Prog" directly or does it just pass everything to %COMSPEC% in the hope that the command interpreter will sort everything out ? How "smart" is it ? How thick a layer on top of the actual underlying operating system API is it ?

» JdeBP «

--- FleetStreet 1.22 NR
* Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

Moin Eddy!

15.03.2001, Eddy Thilleman wrote a message to Vitus Jensen:

(please pardon my late answer, we had CeBIT here in Hannover)

Borland Pascal as DOS .EXE file and compiled it with Virtual
Pascal/2 v2.1 build 243 as OS/2 .EXE file, and implemented it
also in REXX.

...

I've run it without screen output from all the sweep variants
themselves, the only screen output is by the command run in each
(sub)directory. The command in the compiled OS/2 version is run
somewhat slower.

There are larger gaps between execution in the different directories?

Yes.

That would mean that searching for directories is slower in VP/2
which is not to be expected or the startup of every command line
takes longer (s.b.).

I'm not sure about this.

...

The DOS pascal version is equally fast as the REXX version, the OS/2
VP/2 version is slower (as I said).

...

Any ideas?

I tried your Rexx code and translated the pascal version to C. When run both versions in my source tree on the file server (413 dirs) the results are:

timex sweep.cmd -Q cmd.exe /C cd
Elapsed time: 26.67 seconds

timex sweep.exe -Q cd
Elapsed time: 18.68 seconds

So rexx is slower over here. As it is different on your machine it has to do something with VP/2.

BTW, here is the code:

=====================<start>================================
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <direct.h>

#define INCL_DOS
#include <os2.h>


static int fQuiet = 0;
static char szPrgName[_MAX_PATH];
static char szCmdLine[_MAX_PATH] = "";


static void
AllDirs(void)
{
int err;

if( !fQuiet )
{
char pad[_MAX_PATH];
_getcwd(pad, _MAX_PATH);
printf(">>> SWEEP >>> %s\n", pad);
}

err = system(szCmdLine);
if( err == 0 )
{
HDIR hdir = HDIR_CREATE;
FILEFINDBUF3 fbuf;
ULONG cnt;
APIRET rc;

cnt = 1; /* one at a time */
rc = DosFindFirst("*", &hdir,
MUST_HAVE_DIRECTORY
| FILE_ARCHIVED | FILE_DIRECTORY
| FILE_SYSTEM | FILE_HIDDEN | FILE_READONLY,
&fbuf, sizeof(fbuf),
&cnt,
FIL_STANDARD);
if( rc == 0 )
{
do
{
if( !(strcmp(fbuf.achName, ".") == 0
|| strcmp(fbuf.achName, "..") == 0) )
{
chdir(fbuf.achName);
AllDirs();
chdir("..");
}
cnt = 1;
rc = DosFindNext(hdir, &fbuf, sizeof(fbuf), &cnt);
}
while( rc == 0 );
DosFindClose(hdir);
}
} /* end[if(err==0)] */

return;
}


int
main(int argc, char * argv[])
{
int i;

strcpy(szPrgName, argv[0]);
while( argc > 1 && (argv[1][0] == '-' || argv[1][0] == '/') )
{
switch( argv[1][1] )
{
case 'q':
case 'Q':
fQuiet = 1;
break;

default:
fprintf(stderr, "%s: unknown arg \"%s\"\n", szPrgName, argv[1]);
return -1;
}
--argc;
++argv;
}
for( i = 1; i < argc; ++i )
{
strcat(szCmdLine, argv[i]); /* Bug, I know */
}

printf("command: %s\n", szCmdLine);
AllDirs();

return 0;
}
======================<end>=================================



Bye,
Vitus

PS: you don't do a FindClose in your pascal code.

---
* Origin: This tagline does not require Microsoft Windows (2:2474/424.1)

Hello Jonathan,

Sunday 01 April 2001 01:35, Jonathan de Boyne Pollard wrote to Eddy Thilleman:

uses dos;

JdBP> I'm very suspicious of a library of supposed wrappers around system
JdBP> API functions that

...snipped...

JdBP> Try rewriting your Pascal application to call the
JdBP> DosFind{First,Next,Close} functions in the OS/2 system API directly
JdBP> (if such is possible in Virtual Pascal) rather than using this "dos" JdBP> library, whatever it is.

This should be possible.

I'll look into it, thanks.

Exec( Prog, Parm );
if DOSError = 0 then

JdBP> I'm also suspicious of a function called "Exec" that takes so few ...snipped...
JdBP> that the command interpreter will sort everything out ? How "smart" JdBP> is it ? How thick a layer on top of the actual underlying operating JdBP> system API is it ?

I haven't looked inside all this (yet), so I don't know (yet), I'll look into this when I've time.

I also need to write software to automate to post messages (to replace SqTool),
to unpack any archive file (automatically detecting if a file is an archive at all, and if then which archive), and more, more, and more functions I need to write software for. So many things to do, so *little* time. :-(

One thing at a time.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... You can tell a real programmer by the keyboard dents in his forehead.
--- GoldED/2 3.0.1
* Origin: Windows is just a big virus. (2:280/5143.7)

Hello Vitus,

Sunday 01 April 2001 11:53, Vitus Jensen wrote to Eddy Thilleman:

(please pardon my late answer, we had CeBIT here in Hannover)

Have you visited the CeBIT? I have read it was the largest CeBIT ever.

So rexx is slower over here. As it is different on your machine it
has to do something with VP/2.

It's possible. I'll look at doing the API calls directly, when I've time.

My system runs OS/2 Warp v4 fixpak 9. Yours?

PS: you don't do a FindClose in your pascal code.

Jonathan said that too. :) I forgot it, but I'll include in my version with the
direct API calls. When I've converted the pascal version to the API calls, I'll
post that new version and include the results.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... Warp 3, Scotty... and close those damn Windows!
--- GoldED/2 3.0.1
* Origin: C/C++: So simple a child could do it? Go find me a ch (2:280/5143.7)

Moin Eddy!

03.04.2001, Eddy Thilleman wrote a message to Vitus Jensen:

Sunday 01 April 2001 11:53, Vitus Jensen wrote to Eddy Thilleman:

(please pardon my late answer, we had CeBIT here in Hannover)

Have you visited the CeBIT? I have read it was the largest CeBIT ever.

The company where I work daytime (Hoeft & Wessel AG) had a booth as every year.
But this time a demonstration with a new product was shown . And ... it did not work.
I did the OS (porting) and application. A small computer to be installed in a truck, collecting data from a handheld and transmitting it to an FTP server via
GSM. So any problem in the setup involved in one way or another my small compuler. :-(

So rexx is slower over here. As it is different on your machine it
has to do something with VP/2.

It's possible. I'll look at doing the API calls directly, when I've
time.

My system runs OS/2 Warp v4 fixpak 9. Yours?

Warp 4 FP 6.

PS: you don't do a FindClose in your pascal code.

Jonathan said that too. :) I forgot it, but I'll include in my version with the direct API calls. When I've converted the pascal version to
the API calls, I'll post that new version and include the results.

When you forget the DosFindClose it won't hurt unless you try it on a really deep directory structure. How deep? I did not find any documention on the toolkit...


Bye,
Vitus

---
* Origin: Windows:(n.)4. Proof that God has a sense of humor. (2:2474/424.1)

Hello Vitus,

Thursday 05 April 2001 22:09, Vitus Jensen wrote to Eddy Thilleman:

The company where I work daytime (Hoeft & Wessel AG) had a booth as
every year. But this time a demonstration with a new product was
shown . And ... it did not work. I did the OS (porting) and
application. A small computer to be installed in a truck, collecting
data from a handheld and transmitting it to an FTP server via GSM. So
any problem in the setup involved in one way or another my small
computer. :-(

That's really unfortunelately.

My system runs OS/2 Warp v4 fixpak 9. Yours?

Warp 4 FP 6.

That is a difference that also - in theory - could cause speed difference, but I really don't know if that's the case here.

When you forget the DosFindClose it won't hurt unless you try it on a really deep directory structure.

I only tried it on a not so deep directory structure (only 4 levels deep).

How deep? I did not find any documention on the toolkit...

OK.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... OS/2: Bill Gates' worst nightmare!
--- GoldED/2 3.0.1
* Origin: I prefer a dose of common sence. * McCoy (2:280/5143.7)

I also need to write software [...] to unpack any archive file (automatically detecting if a file is an archive at all, and if
then which archive), [...]

Actually, I wrote something similar to this many years ago, but didn't publish it as part of the CLU:

[C:\]unqwk
Incorrect number of arguments.

UNQWK V1.00 Unpack QWK archives using PKUNZIP style arguments.
Copyright (c) 1993 Jonathan de Boyne Pollard General Public Licence v1.00.

UNQWK [Archiver options ...]

UNQWK uses the first non-option argument as the archive name.

Where the options are ('+' means currently enabled) :
/c- Extract to console. /t- Test archive integrity.
/d- Extract using full pathnames. /f- Extract newer files.
/n- Extract new and newer files. /o- Overwrite existing files.
/v- View contents of the archive. /s Unscramble with password.
/p Extract to printer.
/j Apply attribute mask.

[C:\]

Later, for MishMugs, I wrote a REXX script to unpack ARCmail. It which was easier to maintain and to update for new archive types. And because unlike the
above it was single-purpose ("Unpack files matching wildcard A from archive B into directory C."), I didn't have to worry about option processing, which is tricky to do in REXX.

» JdeBP «

... Dulcula, dulcula, dulcula, amorem in ventre habeo.
--- FleetStreet 1.22 NR
* Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

Hello Jonathan,

In april, I posted my little sweep.pas source code to which you commented. I've
now made a new sweep.pas which addresses (some) of your comments.

I call now DosFindFirst, DosFindNext and DosFindClose instead of FindFirst and FindNext; and replaced the call to Exec by a call to SysExecute, because Virtual Pascal comes with full source code of its runtime code enabling me to copy the code inside the Exec procedure to my program. This code calls SysExecute of which I've the source code so I can see what's doing, but I'm not
pretending I understand it (maybe after much studying it). I call the command interpreter (%COMSPEC%) to execute the command in each subdirectory.

This new version of sweep has the same small delays as the previous one, so these delays might be caused by calling the command interpreter (CMD.EXE on my system). The compiled exe file of this new version of sweep is a little over twice the size of the exe file of the previous version to which you commented. If you want, I'll post the source code of this new sweep version.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... Never mind the bloat or quality, look how quickly it was produced!
--- GoldED/2 3.0.1
* Origin: CLICK HERE to play embedded .WAV file (1:2320/38.3)

JdBP>> 1. Change your calls to DosFind{First,Next} to grab many
JdBP>> directory entries in a single call, rather than just a single one
JdBP>> (and then have an inner loop that iterates over the buffer entry
JdBP>> by entry).

Do I need to change or replace the lines with DosFindFirst() and

DosFindNext()

or should I just change the number of entries to find? Problably I need

extra

calls to DosFindFirst() and DosFindNext() to iterate over entries in the

buffer?

How would I make them work with the entries in the buffer?

Use a big buffer. I use 4KiB most of the time, but that's just an arbitrary choice on my part. (Many people make the mistake of using a FILEFINDBUFn structure for the buffer. Just use an array of "char".) Once you have populated the buffer with a call to DosFindFirst() or DosFindNext(), enter an inner loop that iterates over all of the FILEFINDBUFn structures in the buffer.

------------------------------------------------------------------------------------------
#define BSIZE 4096

char buf[BSIZE] ;
HDIR handle = HDIR_CREATE ;
const unsigned long wanted = BSIZE / (sizeof(FILEFINDBUF3) - CCHMAXPATHCOMP + 16) ;
unsigned long count = wanted ;

if (NO_ERROR == DosFindFirst(wildcard, &handle, attributes, buf, BSIZE, &count,
FIL_STANDARD)) {
do {
const FILEFINDBUF3 * fb = (const FILEFINDBUF3 *)(char *)buf ;
while (count--) {
// Do something with *fb
fb = (const FILEFINDBUF3 *)((char *)fb + fb->oNextEntryOffset) ;
}
} while (count = wanted, NO_ERROR == DosFindNext(handle, buf, BSIZE, &count)) ;
DosFindClose(handle) ;
} ------------------------------------------------------------------------------------------

» JdeBP «

--- FleetStreet 1.22 NR
* Origin: JdeBP's point, using Squish <yuk!> (2:257/609.3)

Hello Jonathan,

Tuesday 31 July 2001 08:06, Jonathan de Boyne Pollard wrote to Eddy Thilleman:

This new version of sweep has the same small delays as the
previous one, so these delays might be caused by calling the
command interpreter (CMD.EXE on my system).

JdBP> Possibly, although it's always better to profile the application to
JdBP> determine this. With a REXX script run within the command interpreter JdBP> process, there isn't the overhead of invoking an additional instance JdBP> of the command interpreter when one issues an external command, which JdBP> may be why your REXX script is faster.

I don't have experience profiling with REXX. I'll look into this when I've time.

JdBP> You can optimise things slightly, of course. Here are some
JdBP> suggestions:

JdBP> 1. Change your calls to DosFind{First,Next} to grab many directory JdBP> entries in a single call, rather than just a single one (and then have JdBP> an inner loop that iterates over the buffer entry by entry).

Do I need to change or replace the lines with DosFindFirst() and DosFindNext() or should I just change the number of entries to find? Problably I need extra calls to DosFindFirst() and DosFindNext() to iterate over entries in the buffer? How would I make them work with the entries in the buffer?

JdBP> 2. Obtain the value of the COMSPEC environment variable just
JdBP> once.

I already do this in all the versions.

JdBP> 3. Use the API of my 32-bit CMD to invoke commands, rather than
JdBP> spawning an instance of the command interpreter as a child process.
JdBP> Commands would be interpreted by an interpreter for the CMD language JdBP> running directly in your SWEEP process itself.

I need to study this, I haven't had the time yet (I don't know when I've the time).

JdBP> If the problem is the process execution overhead for the command
JdBP> interpreter, as you suspect, the third suggestion may result in the
JdBP> largest speed increase.

I'll look into it.


Greetings -=Eddy=-

email: e.thilleman@freeler.nl
e.thilleman@hccnet.nl

... Beer Bong? Now that's a sport I can get into!
--- GoldED/2 3.0.1
* Origin: "DOS for dummies?" Isn't that what Windows is? (1:2320/38.3)