Go Wiki/CMS
Thinking of rolling my own - bad idea, but hey, it's what I do. This is closely related to the Immortal Object Server
Core
- Run on NoSql engines (I want to build it out so it can handle a few by default).
- Security by default - if no ssl, uses unauthenticated ssl
- The core is halfway between a wiki and a CMS
I'm going to base the core data structure off the work I did for the [Immortal Object Server]. The "core app" is simply a mark-down engine.
Versions
We will not (at least not initially) support a complex versioning system. However, it will track linear history and who made the changes.
Queries
I want to core of the engine to center around data, therefore being able to execute useful queries is critical. The primary method or organization is tagging. Tagging data puts it in a set. The basic form of query is based on set theory - union, intersection.
The first complication to this is set tagging, or subsets. This says that everything tagged B is also tagged A (though not everything tagged A is B). Therefore B is a subset of A. To make this useful we will want to bring in subset queries. This is accomplished with subset indexing. So A would be everything A (including all of B) but A[0] would only be things actually tagged A. A[1] would be things tagged A via one level of inheritance and A[0,1] would be anything tagged A directly or inheriting A via 1 hierarchy.
Another complication is creating artificial tags via searches.
The goal is something just shy of a database. Users should be able to throw data and have it organized fairly efficiently without defining tables and indexes. There are only two data types text and blob. Text is indexed by word, blob has a text record associated with it, but the data itself is not indexed.
Queries and requests will be separate things. A query will return a set of ids that match the conditions. A request will return a resource (either text or blob). For blobs, there will need to be two requests, one for the meta-data, the other for the blob.
Types of Indexes
- Range
- Done as a key,value pair.
- Allows for >, <, = comparison. Keeps something like a b-tree.
- Contents
- Indexes the occurrence of words in a document allowing for "web like" searches. It would be nice to include a fuzziness bound, something like a Levenshtein distance.
- Hierarchical/Polymorphic Tagging
- Creates relationships where all B's are also A's (but not all A's are B's) so get all A also returns all B
- Syntax is like array, but the bounds are generations, where A is the same as A[:], A[0] is all A but not through inheritence, A[1] is all children of A, A[0:1] is all direct A and children of A.
Symbols
- +: union
- &: intersection
- -: difference
No order of operations, performed left to right.
entrees + base & healthy
Apps
Following my fundamental belief, apps are divided into schemas, services, libraries and UIs. I also want to follow the WordPress model of having a core that does a single thing well and the system really gains power by using a set of apps.
Some types of apps I would like to support early on
- webpages (like wordpress)
- wiki
Groups and Permissions
Groups With Special Membership
- Users
- Groups
- All (users + anon)
Groups With Special Permissions
- Super Users
- Create Users
- Create Groups
Group Permissions
- Exists
- List/read
- Self-admin
- Sub-group
- Admin
How This Works
By setting "list/read" permissions to "all" on a document, every one can view it. By setting "create user" to "all", anyone can create a user (sign up), but restricting it to some group (like "site admins") now only members of that groups can add users.
Hard Problems
If groups can have groups as members how do we find all the members in a single query? Perhaps we don't!
For tags, if a tag can have more than one ancestor, how do we search find all children in one query?
Additional Thoughts
In the long run, this may be a good basis for the Snowing Net servers.
7/1/2016 Notes
Been thinking about this more. I'd like something to work as something Wiki like, but handling both Wiki pages and documents (via ODF JS tools). I also want a file editor.
But what makes sense to me is to provide a core that handles users, permissions, http and storage and provides a plug-in structure. I'm thinking that other applications can register and will be called over either StdIn or named pipes.
8/11/2016 Notes
Continuing from above, same idea but local ports - they work everywhere. I'll build the server to handle http, udp, and websockets. I can extend others later.
Some services are started by the core (if not, there's no specification about how they're started). Those services will need to request that they be started, and possibly when.
Upon startup, a service needs to register with core. First it needs to send a connection request, then negotiate a handshake then send it's configuration details; what domains it's claiming, what traffic types it wants, if it accepts local connections.
Local services
A local service, well provides a service for local use. For instance, there will be a users service that will handle user credentials and permissions.
I'll probably also make a blob-pool service. Things like wiki pages can be stored there. This is specifically intended for sharing data between applications; multiple engines could support a single wiki format.
Services
- ping: just handles http ping requests, included by default to help confirm that a server is up
- auth: manages users, groups and permissions
- admin: administration tools for the server, install services, reboot
- wiki: view and edit pages, that's where this whole thing started
- terminal: uses a websocket to manage a terminal connection.
- dev-tools: code editor, projects, build tools, test tools.
- file: read and write files. They will be organized into different "roots" each containing a directory structure.
- persist: convenient tools for storing structured and semi-structured data. I'll probably end up using Gob/Bolt, but Gob doesn't deal well with revisions, so maybe I'll write a protobuf wrapper.
Clusters
I'd really like this to support clusters. That would be done at the core level. But all this would do is create a list of cores. Each service would need it's own clustering strategy.
A service is either going to request to run as a single instance or as a cluster. If the service runs as a single instance, all requests will be routed to that instance.
If a service runs as a cluster, it will need to request a load balancing strategy. The instance will also need routes to communicate.
Conventions
- Local Service: interface only intended for other services in the cluster to consume
- Public Service: data interface for external consumption, public doesn't necessarily mean public access
- Application: a service with web UI
- Sub-service: A service that registers with another service (or sub-service) rather than the core. This allows for highly flexible services.
We'll need a management and configuration standard. It would be nice for the admin plugin to supply this.