Hey

I was just wondering how you store messages in Clearing Houz? I think
from memory you use couchdb or cockroach db or something?

Is that using a similar format to JSON?

I'm going to experiment with MongoDB on my rust BBS, and wondered if
there was a way to transmit messages in JSON format.

I don't really want to implement FTN, as this BBS is for experimenting,
but I thought it might be fun to experiment with building a new sort of "network".

I was wondering if something like clearing houz would be able to easily
gate messages between FTN and whatever?

Anyway, I've not thought this through very much yet, just tossing out
ideas of things that might be fun to do (My son is out for the day with
his mum visiting poppy - so I have some free time ;P)

Andrew


--- Talisman v0.43-dev (Windows/x64)
* Origin: Smuggler's Cove - Private BBS (21:1/182)

Re: Clearing Houz
By: apam to deon on Fri Oct 07 2022 04:05 pm

Howdy,

I was just wondering how you store messages in Clearing Houz? I think
from memory you use couchdb or cockroach db or something?

So I started Clearing Houz using mongodb only because, I think like you, I wanted to check it out. For me it was problematic because it doesnt enforce a schema (it's benefit) and I found I was spending too much time trying to fix problems, only to find I had a typo in the schema or query here and there. (And probably because I dont know it natively like SQL, so learning how to query it outside of clrghouz was a learning curve I was too impatient to learn.)

I also wanted the replication - one of the designs I had for Clearing Houz, so that nodes could drop mail off (and pick it up) from any place. I think mongo can do, but I had to reasearch how to do it.

So I did switch to CockroachDB - which is SQL and uses the postgres protocol (and in doing so, I discovered an emabarrising amount of mistakes that I had made :) It provided the geo replication that I was wanting out of the box and it is very easy to use (probably because I know SQL better).

(Meatlotion provided a couple of nodes for cockroach - so when you were hitting the website (and hub 3 was tossing mail) it was actually updating the DB that was spread between AU and UK.)

I've put Clearing Houz to sleep for a bit, mainly because I'm busy with other things and because I need to address a latency issue that I thought was making the web experience not great. (I wanted to enable sysops to configure, sign up and almost be able to self-diagnose why their mail wasnt flowing via it). So I may end up ditching cockroachdb if I cant fix that... :(

Is that using a similar format to JSON?

Mongo does return (and I guess does accept) JSON data when querying and storing. I was actually using via an SQL like interface (Clearing Houz is built on laravel - and it has a query interface called eloquent that abstracts the database store).

I'm going to experiment with MongoDB on my rust BBS, and wondered if
there was a way to transmit messages in JSON format.

I don't really want to implement FTN, as this BBS is for experimenting,
but I thought it might be fun to experiment with building a new sort of "network".

I was wondering if something like clearing houz would be able to easily
gate messages between FTN and whatever?

That was another one of my goals with Clearing Houz - given that I had messages in "a database", writing different "packet" formats should be very easy - and I had thought of enabling an API to get/put echomail using a POST with a JSON payload. In my case I'd consume the JSON payload before it got to the DB, but in terms of sharing data around, I do think its a good any easy format to use. (I do a lot of APIs and JSON payloads.)

Anyway, I've not thought this through very much yet, just tossing out
ideas of things that might be fun to do (My son is out for the day with
his mum visiting poppy - so I have some free time ;P)

Awesome, treasures those moments, they seem to get smaller every day !

If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another target...


...δεσ∩
--- SBBSecho 3.15-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another
target...

Thanks for the reply, I didn't realize you'd turned clearing houz off, I understand though, only so many hours in the day. I did think it was a
very cool idea though.

Right now, this is just random ideas in my head.. not sure if they'll work
or not. I'll let you know if I come up with something more tangible.

Andrew


--- Talisman v0.43-dev (Windows/x64)
* Origin: Smuggler's Cove - Private BBS (21:1/182)

apam wrote (2022-10-07):

Hey

I was just wondering how you store messages in Clearing Houz? I think
from memory you use couchdb or cockroach db or something?

Is that using a similar format to JSON?

I haven't used couchdb for quite some time, only for trivial stuff and never in a cluster. Replication is easy to setup and it's all JSON* and Javascript.

*fun fact: when couchdb was alpha 0.something and they used XML as their document format, I proposed they should support or switch to JSON. Nowadays I don't like JSON very much.


⁂

---
* Origin: War is Peace. Freedom is Slavery. Ignorance is Strength. (21:3/102)

On 07 Oct 2022 at 04:05p, apam pondered and said...

I don't really want to implement FTN, as this BBS is for experimenting, but I thought it might be fun to experiment with building a new sort of "network".

If you do build something and would like to test stuff I would be keen to help.

Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

--- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

If you do build something and would like to test stuff I would be keen
to help.

Sure, I'll let you know if I come up with something.

I'm not very inventive.

I was playing with MongoDB, but am thinking it might be simpler to use Postgresql instead. It's not really a network though.

The idea of what I want to do, is a central repository of messages,
stored in a database.

BBSes query the database via an API, it would be something like:

https://some.host/getmsg/XXXXXX

where XXXXXX is the last query id.

the server responds with a json array of messages with an id higher than
the last query id.

to post a message it would send a POST query with a json message (and an
api key for the bbs)

each message has a random uuid so the bbs knows not to import messages
that it exported

oh well screaming baby

andrew


--- Talisman v0.43-dev (Windows/x64)
* Origin: Smuggler's Cove - Private BBS (21:1/182)

Re: Clearing Houz
By: apam to deon on Fri Oct 07 2022 05:14 pm

Hey Apam,

If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another
target...

Thanks for the reply, I didn't realize you'd turned clearing houz off, I understand though, only so many hours in the day. I did think it was a
very cool idea though.

Right now, this is just random ideas in my head.. not sure if they'll work or not. I'll let you know if I come up with something more tangible.

Not sure if you are still playing with the idea of a URL / json kind of packet for data - but if you wanted to try something, then I'm happy to work on it with you.

I've had a bit of energy of late (and time) to work a bit more on clrghouz - and I'm really hoping to get it in a state where it could be Hub 3.

In parallel, it would be easy for me to create a URL endpoint that provides and accepts mail in another format (eg: json).


...δεσ∩
--- SBBSecho 3.15-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

Not sure if you are still playing with the idea of a URL / json kind
of packet for data - but if you wanted to try something, then I'm
happy to work on it with you.

Not really. I'd like to do something, but not much motivation at present.

Cool to see you're working on it again. I thought it was a really neat
project.

Andrew


--- Talisman v0.46-dev (Windows/x64)
* Origin: Smuggler's Cove - Private BBS (21:1/182)

Re: Clearing Houz
By: apam to deon on Wed Nov 30 2022 11:00 pm

Not really. I'd like to do something, but not much motivation at present.

Fair enough - when you get your motivation back let me know...

Cool to see you're working on it again. I thought it was a really neat project.

Thanks :)


...δεσ∩
--- SBBSecho 3.15-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)