Hello folks,

I'm rather stuck on something. Is there a way to read an XML file using the Synchronet Javascript model?

JSON has the lovely JSON.parse(), but I'm not sure how to read the data, or maybe convert it into JSON to make things easier. I even noticed that XMLHttpRequest is not defined, while HTTPRequest is defined.

A lot of the code snippets I find tend to use things in the DOM and has things like hasChildNodes which is not defined in Core JS or Sync.

So, I was wondering if anyone else has tried to do this and has a solution, or at least some advice to give.

For reference, I am trying to grab the data found here: http://www.hamqsl.com/solarxml.php

If I can turn into something usable by Sync, that would be awesome.

~KenDB3

---
■ Synchronet ■ KD3net-Rhode Island's only BBS about nothing. http://bbs.kd3.us

I'm rather stuck on something. Is there a way to read an XML file using the Synchronet Javascript model?

E4X (ECMAScript for XML, IIRC) is still available in Synchronet's JS interpreter as far as I know. It's kinda shitty but can get the job done. Documentation can be found on the web.

There are probably a few Synchronet-specific examples of how to use it; 'exec/load/rss-atom.js', which I made, is not the best but does what it does and should be easy to follow. It uses E4X to turn an RSS or Atom feed (both are XML) into an easier-to-work-with JS object.

JSON has the lovely JSON.parse(), but I'm not sure how to read the data, or maybe convert it into JSON to make things easier. I even noticed that XMLHttpRequest is not defined, while HTTPRequest is defined.

XMLHttpRequest is something that exists in browsers, and doesn't necessarily have to have anything to do with XML. It's a way for a script to make the browser load something asynchronously / in the background via HTTP; good for updating content on a page without the user having to reload.

HTTPRequest is defined if you load 'exec/load/http.js' into your script.
That's an HTTP(S) client for Synchronet's JS environment which you can use in your scripts (and I assume you already do in that weather script).

A lot of the code snippets I find tend to use things in the DOM and has things like hasChildNodes which is not defined in Core JS or Sync.

Yes, if you look for XML parsing via JS, the vast majority of examples will be DOM-based and/or browser specific. We don't have the DOM around here. It's possible that you'd be able to find a pure-JS parser if you looked, which may or may not be portable for Synchronet with some effort.

For reference, I am trying to grab the data found here: http://www.hamqsl.com/solarxml.php

There's already some code for dealing with this exact feed, but it's embedded in an IRC bot. You might be able to find something useful in: '/exec/ircbots/ham/ham.js', however this should work:

load('http.js');
try {
var solardata = new XML(
(new HTTPRequest()).Get(
'http://www.hamqsl.com/solarxml.php'
).replace(
/<\?[^?]*\?>/g, ''
)
).solardata;
} catch (err) {
log('Shit done borked! ' + err);
}

(See https://bbs.electronicchicken.com/temp/solar.txt if that didn't come through okay.)

You should then be able to get at the values from the feed like so:

print(solardata.sunspots);
print(solardata.solarwind);

And so on.

The IRC bot code mentioned above has examples of how to deal with the nested 'calculated(vhf)conditions' values, which gets deeper into E4X than I care to do right now.

Hope this helps.

---
echicken
electronic chicken bbs - bbs.electronicchicken.com - 416-273-7230
■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com

Re: Synchronet Javascript and reading XML
By: echicken to KenDB3 on Wed Jul 27 2016 11:10 pm

Thank you EC, this helps tremendously. It's hard to know what you don't know sometimes.

I'm rather stuck on something. Is there a way to read an XML file
using the Synchronet Javascript model?

E4X (ECMAScript for XML, IIRC) is still available in Synchronet's JS interpreter as far as I know. It's kinda shitty but can get the job done. Documentation can be found on the web.

There are probably a few Synchronet-specific examples of how to use it; 'exec/load/rss-atom.js', which I made, is not the best but does what it does and should be easy to follow. It uses E4X to turn an RSS or Atom feed (both are XML) into an easier-to-work-with JS object.

I will definitely give that a look. I was trying to think of what might deal with XML and for the life of me couldn't remember this one.

JSON has the lovely JSON.parse(), but I'm not sure how to read the
data, or maybe convert it into JSON to make things easier. I even
noticed that XMLHttpRequest is not defined, while HTTPRequest is
defined.

XMLHttpRequest is something that exists in browsers, and doesn't necessarily have to have anything to do with XML. It's a way for a script to make the browser load something asynchronously / in the background via HTTP; good for updating content on a page without the user having to reload.

Ahhhh! Gotcha.

HTTPRequest is defined if you load 'exec/load/http.js' into your script. That's an HTTP(S) client for Synchronet's JS environment which you can use in your scripts (and I assume you already do in that weather script).

I do use it there, and now, thanks to the explanation, I get the concept better than I did before. Much appreciated :-)

A lot of the code snippets I find tend to use things in the DOM and
has things like hasChildNodes which is not defined in Core JS or Sync.

Yes, if you look for XML parsing via JS, the vast majority of examples will be DOM-based and/or browser specific. We don't have the DOM around here. It's possible that you'd be able to find a pure-JS parser if you looked, which may or may not be portable for Synchronet with some effort.

I did a bunch of looking and at first I wasn't turning up much, but eventually I found a pure-JS that would do XML to JSON, but the output wasn't the best. It can be found here for anyone else interested though: http://www.thomasfrank.se/xml_to_json.html

For reference, I am trying to grab the data found here:
http://www.hamqsl.com/solarxml.php

There's already some code for dealing with this exact feed, but it's embedded in an IRC bot. You might be able to find something useful in: '/exec/ircbots/ham/ham.js', however this should work:

load('http.js');
try {
var solardata = new XML(
(new HTTPRequest()).Get(
'http://www.hamqsl.com/solarxml.php'
).replace(
/<\?[^?]*\?>/g, ''
)
).solardata;
} catch (err) {
log('Shit done borked! ' + err);
}

(See https://bbs.electronicchicken.com/temp/solar.txt if that didn't come through okay.)

You should then be able to get at the values from the feed like so:

print(solardata.sunspots);
print(solardata.solarwind);

And so on.

That's awesome. I really appreciate the help!

The IRC bot code mentioned above has examples of how to deal with the nested 'calculated(vhf)conditions' values, which gets deeper into E4X than I care to do right now.

Understood!

Hope this helps.

Immensely. Thanks again for your help EC. Not sure if I can do what I am setting out to do, but it's always fun playing around with stuff.

~KenDB3

---
■ Synchronet ■ KD3net-Rhode Island's only BBS about nothing. http://bbs.kd3.us

There's already some code for dealing with this exact feed, but it's embedded in an IRC bot. You might be able to find something useful in: '/exec/ircbots/ham/ham.js', however this should work:

load('http.js');
try {
var solardata = new XML(
(new HTTPRequest()).Get(
'http://www.hamqsl.com/solarxml.php'
).replace(
/<\?[^?]*\?>/g, ''
)
).solardata;
} catch (err) {
log('Shit done borked! ' + err);
}

(See https://bbs.electronicchicken.com/temp/solar.txt if that didn't come through okay.)

You should then be able to get at the values from the feed like so:

print(solardata.sunspots);
print(solardata.solarwind);

And so on.

The IRC bot code mentioned above has examples of how to deal with the nested 'calculated(vhf)conditions' values, which gets deeper into E4X than I care to do right now.

EC, I have another question (or anyone that feels they can answer).

I notice in the IRC bot code that you can get banned for hitting the web page too often.

Would there be a way to bring in the XML data without an HTTPRequest from a local file? Say something I saved from the last successful pull from the live file and defined as:
var localfile = js.exec_dir + "solar.php";
where solar.php was saved from a previous lookup less than an hour ago?

My awful attempt to do this currently looks like this:

if((time() - last_solar_update) > 60*60) {
try {
var solardata = new XML(
(new HTTPRequest()).Get(
'http://bbs.kd3.us/sol/solarxml.php.xml'
).replace(
/<\?[^?]*\?>/g, ''
)
).solardata;
} catch (err) {
log('Shit done borked! ' + err);
}
} else {
try {
var localfile = js.exec_dir + "solar.php";
var localPHPfile = new File(localfile);
localPHPfile.open("r"); // open file with read access
var localXMLfile = localPHPfile.readAll();
var fixedXMLfile = localXMLfile.replace(
/<\?[^?]*\?>/g, ''
);
var solardata = new XML(
fixedXMLfile.solardata);
} catch (err) {
log('Shit done borked! ' + err);
}
localPHPfile.close();
}

Also, if this is simply a crazy thing to try and do, feel free to let me know. localXMLfile keeps coming back as null and I'm not sure where I've gone wrong anymore.

~KenDB3

---
■ Synchronet ■ KD3net-Rhode Island's only BBS about nothing. http://bbs.kd3.us

I notice in the IRC bot code that you can get banned for hitting the web page too often.

Yes, this is something that I recall hearing about in the past. I guess you get x number of hits per day.

Would there be a way to bring in the XML data without an HTTPRequest from a local file? Say

Sure. I would do something like this:

load('http.js');

var url = 'http://www.hamqsl.com/solarxml.php';
var file = system.data_dir + 'solardata.xml';
var age = 43200; // Seconds

// Fetch data via HTTP and write to file specified above
function getSolarData() {
var sd = (new HTTPRequest()).Get(url);
var f = new File(file);
f.open('w');
f.write(sd);
f.close();
}

// Read data from local file and return parsed XML object
function readSolarData() {
var f = new File(file);
f.open('r');
var sd = new XML(f.read().replace(/<\?[^?]*\?>/g, ''));
f.close();
return sd;
}

// Fetch new data if local file timestamp less than file age specified above
if (!file_exists(file) || time() - file_utime(file) > 43200) getSolarData();

// Read the current data on hand
var sd = readSolarData();

// Now start printing out that fascinating solar data and your happy/sad sun face

It would be worth throwing in some try ... catch blocks somewhere in there, because the HTTP load or XML parsing portions may fail for a variety of reasons.

You could also move the data-fetching part into a separate script and run it on a schedule once, twice, or however many times per day that you want, then the user facing script just loads whatever data is in the file that the other script writes to.

---
echicken
electronic chicken bbs - bbs.electronicchicken.com - 416-273-7230
■ Synchronet ■ electronic chicken bbs - bbs.electronicchicken.com

I notice in the IRC bot code that you can get banned for hitting the web page too often.

Yes, this is something that I recall hearing about in the past. I guess you get x number of hits per day.

Would there be a way to bring in the XML data without an HTTPRequest from a local file? Say

Sure. I would do something like this:

load('http.js');

var url = 'http://www.hamqsl.com/solarxml.php';
var file = system.data_dir + 'solardata.xml';
var age = 43200; // Seconds

// Fetch data via HTTP and write to file specified above
function getSolarData() {
var sd = (new HTTPRequest()).Get(url);
var f = new File(file);
f.open('w');
f.write(sd);
f.close();
}

// Read data from local file and return parsed XML object
function readSolarData() {
var f = new File(file);
f.open('r');
var sd = new XML(f.read().replace(/<\?[^?]*\?>/g, ''));
f.close();
return sd;
}

// Fetch new data if local file timestamp less than file age specified above if (!file_exists(file) || time() - file_utime(file) > 43200) getSolarData();

// Read the current data on hand
var sd = readSolarData();

// Now start printing out that fascinating solar data and your happy/sad sun face

It would be worth throwing in some try ... catch blocks somewhere in there, because the HTTP load or XML parsing portions may fail for a variety of reasons.

You could also move the data-fetching part into a separate script and run it on a schedule once, twice, or however many times per day that you want, then the user facing script just loads whatever data is in the file that the other script writes to.

Thanks again EC, it is much appreciated. :-)

~KenDB3

---
■ Synchronet ■ KD3net-Rhode Island's only BBS about nothing. http://bbs.kd3.us

I faced the same quandary when I was putting together Sports Stats. Ultimately I decided to use Python to write my data scraper. It pulls in the XML and converts it to JSON. Once the data is in JSON, then I can parse it using Sync's JS.

////--------------------------------------------------
BiC -=- http://breakintochat.com -=- bbs wiki and blog

---
■ Synchronet

I faced the same quandary when I was putting together Sports Stats. Ultimately I decided to use Python to write my data scraper. It pulls in the XML and converts it to JSON. Once the data is in JSON, then I can parse it using Sync's JS.

I was very close to doing that if I couldn't get the data a different way. I was hoping to make the script more portable for Sync sysops though. So, thanks to echicken, it turned out pretty well.

~KenDB3

---
■ Synchronet ■ KD3net-Rhode Island's only BBS about nothing. http://bbs.kd3.us