Jan,

It appears that there has been a significant misunderstanding. Scott Little
is not proposing to replace the distribution nodelist with an XML nodelist, but
to run an XML based nodelist in parallel with the standard one.

This would harm nobody.

By all means, let him go ahead. If it has sufficient merit it will eventually be taken up by more and more people. That is how Fido grew from the FidoNet Experiment, and I can see nothing against it now. In any case he needs neither consent nor approval from us to do what he proposes.

The only things he might have to be careful of are the various data protection laws, if he proposes to make significant changes in the content of the nodelist.

As an aside it is a little sad that you write far better English than he does. But then I don't speak Strine, either. :-)

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

[ 31 Dec 02 00:24, Bill Birrell wrote to Jan Vermeulen ]

It appears that there has been a significant misunderstanding.
Scott Little is not proposing to replace the distribution nodelist
with an XML nodelist, but to run an XML based nodelist in parallel
with the standard one.

Actually, XML (or HRN or whatever other format) will eventually replace SLF (the current format) on those systems that can handle it/want it - running in parallel is only necessary while the tools are new and 100% redundancy is requred.

But like I said, systems that run on XML will emit SLF nodelists for whoever needs them - that conversion is a simple one.

-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Actually, XML (or HRN or whatever other format) will
eventually replace SLF (the current format) on those
systems that can handle it/want it - running in
parallel is only necessary while the tools are new and
100% redundancy is requred.

So, just do it, but stop frightening the horses!

But like I said, systems that run on XML will emit SLF
nodelists for whoever needs them - that conversion is
a simple one.

And meantime the normal nodelist will still be available from the normal channels. How long meantime lasts is anybody's guess. It's been about fifteen years so far. FidoNet has monumental inertia.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

Jan,

Hi, Bill.

It appears that there has been a significant misunderstanding.
Scott Little is not proposing to replace the distribution nodelist
with an XML nodelist, but to run an XML based nodelist in parallel
with the standard one.

That has become clear now. He was following up the Husky group that threatened to replace unwilling *Cs and it looked like he was contaminated by them.

This would harm nobody.

Probably not.

By all means, let him go ahead.

I can't stop him. But I'll stay wary, as my job requires me to be.

As an aside it is a little sad that you write far better
English than he does.

Don't flatter me, Bill, or my hat will squeeze.

But then I don't speak Strine, either. :-)

I take 'strine' to be short for 'english as prononced by the Ozzees'. Am I right?


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

[ 01 Jan 03 03:13, Bill Birrell wrote to Scott Little ]

And meantime the normal nodelist will still be available from the normal channels. How long meantime lasts is anybody's guess. It's been about fifteen years so far. FidoNet has monumental inertia.

I would say forever. NODELIST and NODEDIFF file echos will always carry SLF, and will always have someone hatch SLF versions into it. XML will use it's own.


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Hey Jan,

I take 'strine' to be short for 'english as
prononced by the Ozzees'. Am I right?

Yes, it's the 'strine' way to say australian. :-)

OK on the rest. While the penny has dropped that these people mean us no harm, and have begun to accept that they may have problems with the coding, it still has not occurred to them that in many places they are arguing in a circle, and in others they are assuming what has yet to be proved. I'll put that down to missionary zeal. ISTR we had much the same thing a decade or so ago with SQL, but that came to nothing, although it was a good idea at the time. That is one reason why we still have the comma-separated (St. Louis Format) SLF plain text nodelist. The other obvious reason is that all programming languages support the format.

Now another very simple question - will the XML or HRN new nodelist be shorter than the current compressed one? If so, it will be cheaper to send on PSTN (or POTS) lines that still are not free of charge. If it is very significantly shorter it might become 'annoying behaviour' not to make it available to downlinks.

Of course I am also assuming here what has yet to be proved - that reliable
software to make XML nodediffs and to make use of them can be produced as freeware. I believe FTS or policy requires that there be at least one Public Domain freeware version. Others than the 'official' one may be shareware.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

[ 02 Jan 03 12:18, Bill Birrell wrote to Jan Vermeulen ]

us no harm, and have begun to accept that they may have problems with
the coding,

Coding will not be a problem at all. The only problem I foresee is broken data
converted from SLF.

to be proved. I'll put that down to missionary zeal. ISTR we had much
the same thing a decade or so ago with SQL, but that came to nothing,

SQL is a storage format not a distribution format.

nodelist. The other obvious reason is that all programming languages support the format.

As with XML.. it's still text

Now another very simple question - will the XML or HRN new
nodelist be shorter than the current compressed one?

No. A directly converted one will be slightly larger compressed, and significantly larger uncompressed (close to twice the size, thanks to all the tags/field names). Once the alternate format is the origin of the data, it will get even bigger as people start putting more information in it.

Of course I am also assuming here what has yet to be proved - that reliable software to make XML nodediffs and to make use of them can
be produced as freeware.

I doubt many are interested in closed source these days..


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Quoting Bill Birrell on Thu 2 Jan 2003 12:18 to Jan Vermeulen:

Now another very simple question - will the XML or HRN new
nodelist be shorter than the current compressed one? If so, it will
be cheaper to send on PSTN (or POTS) lines that still are not free
of charge. If it is very significantly shorter it might become
'annoying behaviour' not to make it available to downlinks.

XML zipped nodelists are approx 30% _larger_; I do not know if this is due to the tags (which are needed in order xml will work) or to a less efficient zip library caused by those tags.

Of course I am also assuming here what has yet to be proved -
that reliable software to make XML nodediffs and to make use of
them can be produced as freeware. I believe FTS or policy requires
that there be at least one Public Domain freeware version. Others
than the 'official' one may be shareware.

I think you're right.


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

Quoting Scott Little on Fri 3 Jan 2003 16:44 to Bill Birrell:

The only problem I foresee is broken data converted from SLF.

'Broken data converted from SLF' can only be caused by:

1. Broken data in the SLF, i.e. data that is broken in such a way
that SLF list compilers and readers can't use them.

Please be more explicit: in what circumstances has this happend?

2. Data being broken while being converted.

You do not mean that there will a problem, do you? I presume that
those doing the conversion have a good understanding of the data
and how to construct the new envelope (after all you will not
change the data itself, will you?).

3. The most unlikely of all: that the data will not work in an XML
environment.

Let's not even think about that one, will we?


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

[ 03 Jan 03 17:49, Jan Vermeulen wrote to Scott Little ]

'Broken data converted from SLF' can only be caused by:

1. Broken data in the SLF, i.e. data that is broken in such a way
that SLF list compilers and readers can't use them.
Please be more explicit: in what circumstances has this happend?

Everywhere - ask David Drummond for a list of broken nodes he's dug up and had to request it be fixed.

2. Data being broken while being converted.
You do not mean that there will a problem, do you?

Unlikely. The conversion will be closely monitored by the developers as long as SLF remains the primary data source, so issues converting ambiguous data will be quickly fixed or worked around.

I presume that those doing the conversion have a good understanding
of the data and how to construct the new envelope (after all you will
not change the data itself, will you?).

The data will be changed to comply with the new format. Anything that cannot be understood will be put in a generic <flags> or <userflags> field.

3. The most unlikely of all: that the data will not work in an XML
environment.
Let's not even think about that one, will we?

The only issues there are buggy or stupidly designed programs..


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Hi Scott,

I would say forever. NODELIST and NODEDIFF file echos
will always carry SLF, and will always have someone
hatch SLF versions into it.

I already said "just do it!" Now another: "Talk is cheap."

Get on with it!

Produce a specimen list of software available for nodes and points, please.
The list should include mailers or BBS or both which can use either IP or FTS protocols or both on one phone line. Also it should include nodelist compilers,
echomail processors, mail readers and point software. At least one complete system should be freeware.

If you can get general agreement on the list of things required, it should not be too hard to develop these into outline specifications, then program specifications, then black boxes or beans, then programs.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

David,

I'm quoting Scott Little on Sat 4 Jan 2003 9:27 to Jan Vermeulen:

'Broken data converted from SLF' can only be caused by:

1. Broken data in the SLF, i.e. data that is broken in such a way
that SLF list compilers and readers can't use them.
Please be more explicit: in what circumstances has this happend?

Everywhere - ask David Drummond for a list of broken nodes he's dug
up and had to request it be fixed.

Could you please give us (an update of) this list?

Thank you.

-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

Quoting Scott Little on Sat 4 Jan 2003 9:27 to Jan Vermeulen:

'Broken data converted from SLF' can only be caused by:

1. Broken data in the SLF, i.e. data that is broken in such a way
that SLF list compilers and readers can't use them.
Please be more explicit: in what circumstances has this happend?

Everywhere - ask David Drummond for a list of broken nodes he's dug
up and had to request it be fixed.

In a separate posting I have done so.

2. Data being broken while being converted.
You do not mean that there will a problem, do you?

Unlikely. The conversion will be closely monitored by the
developers as long as SLF remains the primary data source, so
issues converting ambiguous data will be quickly fixed or worked
around.

Good.

I presume that those doing the conversion have a good understanding
of the data and how to construct the new envelope (after all you will
not change the data itself, will you?).

The data will be changed to comply with the new format. Anything
that cannot be understood will be put in a generic <flags> or
<userflags> field.

How can you be 100% sure that you will get the old data back when generaing
an SLF list from the XML data?

3. The most unlikely of all: that the data will not work in an XML
environment.
Let's not even think about that one, will we?

The only issues there are buggy or stupidly designed programs..

Which will not happen. Is that what you want to say?


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

[ 03 Jan 03 23:23, Bill Birrell wrote to Scott Little ]

Get on with it!

Keep yer pants on!

We're in the planning and design phase - the longest and most boring part of any project. It's also the most critical - if the design is wrong, then everything else will suck (like building your house on sand). Writing the code
is easy, and will happen when the design is near final.

It's also the holidays, so everyone's off seeing the world in meatspace...

The list should include mailers or BBS or both which can use either
IP or FTS protocols or both on one phone line. Also it should include nodelist compilers, echomail processors, mail readers and point
software. At least one complete system should be freeware.

Why do we need to do any of that?


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Quoting Bill Birrell on Fri 3 Jan 2003 23:23 to Scott Little:

Produce a specimen list of software available for nodes and
points, please. The list should include mailers or BBS or both
which can use either IP or FTS protocols or both on one phone line.
Also it should include nodelist compilers, echomail processors,
mail readers and point software. At least one complete system
should be freeware.

If you can get general agreement on the list of things
required, it should not be too hard to develop these into outline specifications, then program specifications, then black boxes or
beans, then programs.

I was going to write a paper on that, but now you did all the work for me ;)

Keep up the good work, Bill.

-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

[ 04 Jan 03 15:26, Jan Vermeulen wrote to Scott Little ]

How can you be 100% sure that you will get the old data back when generaing an SLF list from the XML data?

You can't, but that's dependant on the broken-ness of the input SLF. Theoretically, the SLF -> XML conversion will only extract "known good" data, leaving the rest as undecipherable nonsense which XML native programs will ignore, but will be restored when converted back to SLF.

As I said before, if this is a concern, *Cs can simply run both in parallel. It's likely they won't, though, as it will force nodelist entries that are undecipherable to be fixed instead.

The only issues there are buggy or stupidly designed programs..

Which will not happen. Is that what you want to say?

Never say never :)


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Quoting Scott Little on Sun 5 Jan 2003 6:09 to Jan Vermeulen:

How can you be 100% sure that you will get the old data back when
generaing an SLF list from the XML data?

You can't, but that's dependant on the broken-ness of the input
SLF. Theoretically, the SLF -> XML conversion will only extract
"known good" data, leaving the rest as undecipherable nonsense
which XML native programs will ignore, but will be restored when
converted back to SLF.

This implies that an XML list generated from the nodelist at one place will
not yield the same nodelist somewhere else. I don't like that.

As I said before, if this is a concern, *Cs can simply run both in parallel.

I would not know why I should do that. There's more important work to do than that running wo programs that do the same and then check them for correctnes (and I can't use a simpel "FC first second > watsup"

It's likely they won't, though, as it will force nodelist entries
that are undecipherable to be fixed instead.

I can do that using MakeNL and a flag list; I'm doing that already now ;-)


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

[ 04 Jan 03 22:20, Jan Vermeulen wrote to Scott Little ]

This implies that an XML list generated from the nodelist at one
place will not yield the same nodelist somewhere else. I don't like
that.

It's certainly possible, but unlikely. If the original data isn't broken to start with, there will be no problems.

As I said before, if this is a concern, *Cs can simply run both
in parallel.

I would not know why I should do that.

Then don't. Stay with SLF. It's your loss.

There's more important work to do than that running wo programs that

There's no extra work involved other than setting the new one up.


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

XML zipped nodelists are approx 30% _larger_; I do
not know if this is due to the tags (which are needed
in order xml will work) or to a less efficient zip
library caused by those tags.

With the present FidoNet Policy document (4.07) that is going to be a problem. As things stand the only approvable changes are those which reduce the
size of the nodelist and therefore the cost of distribution.

Would you please confirm that I have interpreted policy correctly? If so, it would seem that the XML nodelist would have to be distributed by internet if
at all. That brings back the ethical problem of the dime that I mentioned earlier. I have personally sidestepped that lemma by keeping FTS and internet entirely separate.

I note with approval that Scott Little takes public domain source for granted. I hope his fellow developers share this view.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

The list should include mailers or BBS or both which can use either
IP or FTS protocols or both on one phone line. Also it should include nodelist compilers, echomail processors, mail readers and point software. At least one complete system should be freeware.

Why do we need to do any of that?

It has to be done. That means somebody has to do it. You are the main protagonist. Why not you? If not you, then delegate it to one of your minions. :-)

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

Hey Jan,

Produce a specimen list of software available for nodes and
points, please. The list should include mailers or BBS or both
which can use either IP or FTS protocols or both on one phone line. Also it should include nodelist compilers, echomail processors,
mail readers and point software. At least one complete system
should be freeware.

Scott Little asked why he should bother. I find the naivety of the question
hard to believe. If you wish to reform you must carry the bulk of the people with you. Otherwise you will fail, sure as "eggs is eggs". To carry a populace,
the easiest way is to spoon feed them - otherwise we are back to "The Prince" by Machiavelli, and we've just prised him away from those methods. :-)

I was going to write a paper on that, but now you
did all the work for me ;)

Keep up the good work, Bill.

I still need to know whether my reading of Policy 4 is the correct one. Other than Ward, I would trust your judgement most on that.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

[ 06 Jan 03 00:25, Bill Birrell wrote to Scott Little ]

either IP or FTS protocols or both on one phone line. Also it
should include nodelist compilers, echomail processors, mail
readers and point software. At least one complete system
should be freeware.

Why do we need to do any of that?

It has to be done.

Why?


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)

Quoting Bill Birrell on Sat 4 Jan 2003 23:24 to Jan Vermeulen:

XML zipped nodelists are approx 30% _larger_; I do
not know if this is due to the tags (which are needed
in order xml will work) or to a less efficient zip
library caused by those tags.

With the present FidoNet Policy document (4.07) that is going
to be a problem. As things stand the only approvable changes are
those which reduce the size of the nodelist and therefore the cost
of distribution.

If you aim at the second paragraph of P4, 2.1.9, you do have a point as it is one place that points out that FidoNet is not the place to cause undue cost to other sysops.

Thinking of that, that is valid for any action one takes in FidoNet, like flagging one's modem as V32B where it is V34, or the opposite.

-=<[ JV ]>=-

Would you please confirm that I have interpreted policy
correctly? If so, it would seem that the XML nodelist would have to
be distributed by internet if at all. That brings back the ethical
problem of the dime that I mentioned earlier. I have personally sidestepped that lemma by keeping FTS and internet entirely
separate.

I note with approval that Scott Little takes public domain
source for granted. I hope his fellow developers share this view.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)
SEEN-BY: 10/3 345 28/1 105/8 360 106/1 1234 2000 123/500 124/5009
5025 140/1
SEEN-BY: 143/2 154/15 201/505 205/1 229/1 247/101 250/99 254/6
275/311
SEEN-BY: 278/230 280/4 17 21 100 282/4066 311/13 322/320 343/41
362/627
SEEN-BY: 379/1 396/45 751/321 2604/416 2624/306 3634/12

* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

Hi Jan.


04-Jan-03 Jan Vermeulen wrote to Scott Little

How can you be 100% sure that you will get the old data back when generaing an SLF list from the XML data

Is it desirable to get 100% the same data back anyway?

things like the ordering of the flags or the method used to publish
internet address aren't critical and some systems work better with
one form and others with a different form.

Of cource we have to see to it that everything we change also can
be provided in a backward compatible format for the sysops.

Ok, the intention is there. But how sure can you be that not even
one byte will get lost or damaged in the operation

One way is to prove the software and specification mathematically,
but first a design is needed.

How can you be 100% sure that you will get the old data back
when generaing an SLF list from the XML data?

You can't, but that's dependant on the broken-ness of the input
SLF. Theoretically, the SLF -> XML conversion will only extract
"known good" data, leaving the rest as undecipherable nonsense
which XML native programs will ignore, but will be restored when
converted back to SLF.

This implies that an XML list generated from the nodelist at one
place will not yield the same nodelist somewhere else. I don't
like that

why? as long as it contains the apropriate information does it matter,


when the extractor has to produce SLF for that nodeline it cant know how
the line was originallt organised, but it can express the information in a sensible way.

suppose this goes in:

,100,213.84.184.65,Wormerveer,Jan_Vermeulen,31-75-6400418,9600,CM,XA,V32B
,V42B,V34,VFC,V120L,V120H,X75,IBN,PING,U,ENC

if it comes out like this:

,100,213.84.184.65,Wormerveer,Jan_Vermeulen,31-75-6400418,9600,XA,V32B,
V34,V42B,VFC,V120L,V120H,X75,IBN,PING,CM,U,ENC

does it really matter?

Bye <=-

---
* Origin: Darth Vader sleeps with a Teddywookie. (3:640/1042)

Quoting Jasen Betts on Sun 5 Jan 2003 22:16 to Jan Vermeulen:

04-Jan-03 Jan Vermeulen wrote to Scott Little

How can you be 100% sure that you will get the old data back when
generaing an SLF list from the XML data

Is it desirable to get 100% the same data back anyway?

It is a must.

things like the ordering of the flags or the method used to publish internet address aren't critical and some systems work better with
one form and others with a different form.

If you're not strict to start with, you create a nursery for bugs.

Of cource we have to see to it that everything we change also can
be provided in a backward compatible format for the sysops.

Ok, the intention is there. But how sure can you be that not even
one byte will get lost or damaged in the operation

One way is to prove the software and specification mathematically,
but first a design is needed.

Ok.

How can you be 100% sure that you will get the old data back
when generaing an SLF list from the XML data?

You can't, but that's dependant on the broken-ness of the input
SLF. Theoretically, the SLF -> XML conversion will only extract
"known good" data, leaving the rest as undecipherable nonsense
which XML native programs will ignore, but will be restored when
converted back to SLF.

This implies that an XML list generated from the nodelist at one
place will not yield the same nodelist somewhere else. I don't
like that

why? as long as it contains the apropriate information does it
matter,

Do not start to get lax even before you got the specs. You're sure not to succeed if you do.

when the extractor has to produce SLF for that nodeline it cant
know how the line was originallt organised, but it can express the information in a sensible way.

suppose this goes in:

,100,213.84.184.65,Wormerveer,Jan
_Vermeulen,31-75-6400418,9600,CM,XA,V32B
,V42B,V34,VFC,V120L,V120H,X75,IBN,PING,U,ENC

if it comes out like this:

,100,213.84.184.65,Wormerveer,Jan
_Vermeulen,31-75-6400418,9600,XA,V32B,
V34,V42B,VFC,V120L,V120H,X75,IBN,PING,CM,U,ENC

does it really matter?

Not in this particular, well chosen example. But surprises tend to happen just when you think you got everything under control.


-=<[ JV ]>=-


* Origin: The Poor Man's Workstation -- Wormerveer NL (2:280/100)

It has to be done.

Why?

Why not?

bb.

---
* Origin: Escan BBS (2:25/200)

Thinking of that, that is valid for any action one
takes in FidoNet, like flagging one's modem as V32B
where it is V34, or the opposite.

Not guilty! :-)

From the list of now discredited examples of 'annoying behaviour' doubing the size of the nodelist would seem to me to be a prime example, but that can be taken up in a policy echo rather than here, perhaps.

Best Wishes,
Bill.

---
* Origin: Escan BBS (2:25/200)

[ 08 Jan 03 02:46, Bill Birrell wrote to Scott Little ]

Why?

Why not?

I don't have to prove a negative.


-- Scott Little [fidonet#3:712/848 / sysgod@sysgod.org]

--- FMail/Win32 1.60+
* Origin: Cyberia: All your msgbase are belong to us! (3:712/848)