Service Oriented Network

From Adam Colton
Revision as of 00:46, 18 September 2014 by Adamcolton (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

First of all, this needs a better name - I'm thinking Snowing Net. Alternatly, ArcNet "The Arc of the Moral Universe Is Long But It Bends Toward Justice".

My thinking on this topic originally arose out of my discontent with services, particularly social networking services, on the internet. I wanted a way that many services could collaborate while simultaneously controlling my data. As I explored the problem, I realized this type of a structure was incompatible with an advertising based economy.

While I was working on this, information about the Prism program came out as well as the incident with Lavabit. I had already envisioned heavy encryption as part of my standard, but this made me want to find a way to build services where providers do not have the keys, that way they could not be compelled to hand them over.

This system is more of an API than an application. It describes a framework for people and servers to interact so that no entity can control or corrupt the system. I believe that a rudimentary version could be established with very little code and over time, new standards can be build to develop a mature system.

For why, see SON: Motivations.

Snowing Technical Operations

Resources

https://www.youtube.com/watch?v=nT-TGvYOBpI

Notes from Lauren

Stop using hyphens.

Include potential examples of metering. Perhaps a seperate examples doc.

Story sounds like a sales pitch.

Include a mitigations section. Perhaps break problems down into mitigated and standing.

Core-Principles

  • Pseudonymity
  • A La Carte
  • Separation of Services and Service Neutrality
  • Protection of Reasonable Copyright
  • Least acceptable privilege
    • Encryption by Default
    • Data is controlled by owner
  • Forward progress is more important than backwards compatibility

Introductory Parable

Before I get into the depth of the system, I wanted to give a high level perspective. I want to do this by way of a parable. In our story Alice is average user and is introducing Bob to the network. I fully admit that it is contrived and optimistic. But it explores the type of network I would like to see.

Alice explains to Bob that this network exists alongside the internet and is accessed right through a browser. When she set it up the first time, she had to install one piece of software, but after that it has been quite easy to use. The only thing she had to learn was 'key management'. She tells Bob that a key is like a password, but it's stored in a file, so you have to be very careful that it's backed up and that no one else has access to it. She tells Bob that there are special programs that can help with this.

Bob watches Alice for a minute as she reads some emails, checks her social network and pulls up a few blogs and news sites she wants to read. After a minute Bob asks, "Where are all the ads? Do you have some ad blocker?"

Alice explains that there are essentially no ads. She's paying to use the system. Bob is initially taken aback by this - being used to free access to data. But Alice tells him that a month of regular usage only costs her a few dollars. Each of the ads on a traditional page is only worth a fraction of a penny. And those ads drive up network resources and compute resources making the very network they support more expensive.

Alice also says that she likes paying for the content - it's a higher quality. The traditional internet sites cater to the advertisers, delivering quality content to her is a lower priority. On this network it's the only priority.

Bob watches for a few more minutes then asks why so many different sources of information are folded seamlessly onto a single page - why doesn't each site have it's own page and it's own style. Alice says that most of the services she uses only provide data, no UI. Instead she uses a service who's focus is UX and UI and that service weaves all the information together in a way that works well for Alice. She tells Bob that there are several services like that each focusing on different types of users - some focus on simplicity and ease of use, others focus on power users, and others make everything highly customizable. But she likes it because services don't waste time building a UI and the UI focuses on building the best experience for users.

Bob asks Alice why she started using the network. She tells him that she a few friends convinced her. One of them was really concerned with security and she liked it because she could see that all her data and all her communications were encrypted. Another was sick of rebuilding the same social network every two years on a new site and found that data about his social network belonged to him, if he ever changed what app he did his social networking with, he could take the network with him. This also meant he could interact with anyone on this network, regardless of what social networking service they used. Another friend had been working on an app for the network and was really excited because she didn't have to get approval to put her code in a store, but she also didn't have to set up a server. Just by making the application available servers would host it when people requested it and she would get a cut.

Bob immediately got set up and it was the best decision he ever made.

Core Components

  • Local Identity Server
  • Remote Identity Server
  • Data and Application Servers
  • Protocols
  • Repositories
  • Reputation
  • Arbitration

Local Identity Server

This is a small server that sits on the local machine and manages the master keyring.

Remote Identity Server

This is a server that manages valet keyrings for many users. The RIS is the concierge - its single objective is to make interacting with the network easier on users. The RIS is responsible for billing and 'always on' services.

There are a number of ways that billing can be done, but the core idea is similar across all of them. It would be inconvenient for a user to end up owning fractions of a penny to hundreds of DA servers across the net. Instead, the RID server runs up a tab for all it's users with each individual DA server and pays the tab per DA server policy. The RID server then collects the debt from each user for all the tabs it has paid.

A set (as in set theory) way of looking at it is given U user and D DA servers you would end up with U x D transactions. By placing R RID servers in the middle you end up with (U x R + R x D) transactions and as long as R is significantly smaller than U, this should be far fewer transactions. And going back to the original problem - it's much easier on the users because they only have to settle a bill with their RID server.

I speak about payment throughout this document. The majority of this will happen in BitCoin - this is easiest way to transfer wealth digitally. However, I expect that RID servers will charge users in a currency and format they are comfortable with. This way, end users should be completely isolated from BitCoin, unless they seek out a way to pay in that medium.

RID servers should also provide reasonable defaults for users. Any single 'app' will probably rely on dozens of underlying services, which should be configurable by the user. But because most user will not want to deal with all of these details, the RID should give them a solid default. An analogy would the Linux ecosystem - the core spec (what you're reading) is the kernal, the DA servers are all the little command line programs you can call into, the repos are the package system, and the RID is the distro.

Data and Application Servers

DA servers store user data and run applications. These are pretty close to a traditional web server.

Protocols

Protocols define other data types. These are essentially structs (not quite objects, because they don't include methods). Anyone can publish a protocol, but the hope is that the best ones will bubble up and standards will develop preventing too much duplication.

Repositories

Repos store code. Developers put their code in a public repo and a DA server can launch it as a service. Part of the repo is usage specs - how to deploy, what the author expects to be paid, what the copyright status is, and any dependencies. The DA is responsible for paying the author(s).

Reputation

Every component of the system can review every other part of the system. If servers (ID or DA) are engaging in shady behavior, it will effect their reputation. Possibly most important, repositories can be reviewed and endorsed. These reviews can be paid, but payment must be disclosed. The idea is a sort of agency - some individuals can build a reputation reviewing repos, even users who don't understand code can learn to trust these reviews or at least the aggregate of the reviews. Hopefully, the best of these agents will be paid by the ID&DA servers so there is no conflict of interest, though I would also like to see developers pay into a pool.

The interesting structure here is that meta-reviews should be possible - this would be similar to an up-vote or a down-vote. Collectively this should provide enough data to find the entities who give the highest quality reviews (non-biased, naturally following a bell curve, detailed, precise) and produce weighted reputation. Doing this, would be a service, hosted on a DA. This is the sort of interwoven structure I'm looking for.

Arbitration

The protect copyright and enforce payment, arbitration can be called in. This process is meant to be very open. Arbitration can be administered by many people (a jury) or an individual and they may be experianced in arbitration or lay people - the choice is left to those seeking arbitration. Particularly for copyright, it is meant to represent the aggregate position of the network rather than any hard and fast rules. The line between fair use, imitation, parody and infraction is hard to separate, so rather than trying to do so with rigor, we do so by the majority of opinion.

Convention

These are the thing I want to see the network follow and I think are important for its continued existence. However, they cannot be codified and there may be exceptions. My hope is that a small, fervent group will help keep the network honest.

  • Encrypt user data
  • Separation of Service and Ads
  • A la carte
  • Reasonable Copyright Expiration
  • Seperation of UI, logic and schema

Encrypt user data

Unless there is a compelling reason not to, data should be encrypted. Even 'shared' data should be encrypted - if I post a picture and one of my friends wants to see and has the authorization to do so, the image should decrypted by my RIS, and re-encrypted with my friends public key. This is just an example - there are other methods that would work, for instance I could encrypt the image with symmetric encryption and provide the friend with the decryption key rather than re-encrypting for their account. There are many options, and I hope that very smart people will figure out which ones work best. The point is that data should not be stored or transmitted unencrypted unless it is meant for public consumption.

Separation of Service and Ads

When providing a service to users, it should not be coupled with advertisements. Also see a la carte.

A la carte

Users pay a reasonable, price - stated up front - for what they get. Services should be billed on a usage basis. Subscriptions discourage users from signing up. This not to say subscriptions should not be offered, but tiered usage should be discouraged (ie. a free tier and a subscription tier). A service should have 100% functionality at the a la carte level. I would call out the AWS model as a good standard - allowing users to pay an upfront fee for a period of reduced cost.

Reasonable Copyright Expiration

Repos and data can be tagged with meta-data, among the things that can be added is a copyright notice. Copyrights should be respected (and with arbitration this should be enforced) but the corollary is that they should be reasonable, a few years to a decade. I would also really like to see a maximum earning level - creators saying "The most I ever deserve to make off this is X" and after they have earned X, they expire the copyright and place the work in the public domain.

Seperation of UI, logic and schema

Applications should be either UI or Logic. UI should send commands to a logic application. Logic applications can talk to other logic applications and send state changes to data stores that use the schema the logic is built around.

Structures

This are structures I hope or suggest. I'm sure others will arise and that some of these will prove infeasible or inferior to others I have not considered. These are simply meant as a starting point.

Lists and Indexes

References to resources. Helpful for finding DA servers, RID servers, repos - really any thing there exists a protocol for, there should be a list of things following that protocol. I would also like to see a model where the owner of the data registers with the list and the list confirms that it properly follows the schema.

Agency Pools

Pools of agents, formed into an organization to review services and repos. The funding for these pools should be public and diverse to avoid conflicts of interest.

Developer Investments

Individuals or groups can invest in developer projects and get a cut of any profits they may make.

Open Data Standards

Applications should employ open data standards so that other services can collaborate. If one service helps build and maintain a social network, another service may have a use for that data. Users can authorize that other service to mine their social network. Open standards are better for everyone. I also expect their to be conversion services so that if two incompatible standards encoding the same type of data arise, a conversion service could present your copy of the data to a service using the other standard.

The system itself would also be subject to these standards. As flaws are discovered or the needs change, new specs can be suggest and developed by the community. Only by having a significant portion of the network adopt these specs do they become a standard.

Pricing

Everything should be paid for, but mostly in micro transactions. I believe it is inevitable that the behind-the-scenes accounting will get very complex, but it should remain clear to end users what they are paying for.

DA Servers

These are the simplest pricing model, essentially how cloud services already work. Users pay set prices for bandwidth, computation and storage.

Repos

Repos get a little complicated and I'm not sure exactly how they will work. I'm sure the system will come up with several possible solutions. One that I see is that the code includes some accounting so that every time a given action is performed, the cost is tabulated. It's also important to note that if I write something and include someone elses repo in my code (by reference, not by copying) I don't have to worry about paying them, they DA server will see that repos inclusion and pay them their cut accordingly.

Maintainance Payments

This either leaves the authors with no incentive to maintain their code or (by counting each successive release as independent) no realistic path for a project to become paid in full. To alieviate this, specs can be introduced outlining various payment structures. One that I would suggest here is that successive check-ins can continue to be made and after a point (when a feature is complete) pricing, a value and a date is set for that set. This does not change the original pricing, value and date.

For example, say a developer releases a chat application priced at $0.0001/1000characters, valued at $20,000 and with a public release date set two years in the future. Then, over the next six months, successive check-ins add a picture sharing feature. That feature could be independently set to $0.0001/kb, valued at $10,000 with a release two years in the future. 18 months later (or if the $20,000 price is met) the chat features could be used for free, but the picture service would continue to cost a fee for another 6 months (or until the $10,000 is met).

LIDs

Because you run a LID on your own server, you don't have to pay for computation. But you do need to pay for the repos that run on your LID. This will probably be the primary point of piracy. Hopefully this will be kept to a minimum and we can find practical ways to monitor it without infringing on privacy or pseudonimity.

Schemas and Standards

This is the one thing that I intend to be free. There is no cost to schemas or standards. They have to be hosted somewhere, so a DA will get paid, but there unlike repos, the authors cannot claim a fee.

Problems

There are a number of flaws that still need to be overcome before I think this could work.

Dishonest DAs

A DA server could point to a repo but inject their own code. There are two scenarios I see as most problematic. The first is interception of user data. Essentially a reversion to the current state, provide services at a loss and sell the user data acquired at a profit. The second scenario is DAs simply breaking the rules and pointing to a repo that says what the application does, but has no actual source code, or not pointing at a repo at all. This, again, would be reversion to the current system where closed servers run closed code.

Dishonest RIDs

A similar problem, an RID server could be selling user data. Slightly less problematic because the use of valet keys should help track the source of the leak.

Application Trust

This is one part dishonest-DA and one part User-Error. Under this system, an application layer is delivered by the network. Some quantity of trust needs to be granted to the application layer. We can compartmentalize trust (sandboxed disk, unlimited disk, visual/audio, sound, network) to reduce the risk. We can also rely on the RIDs to help protect users (a RID should help manage trust at a high level and inform users if they're about to do something dangerous). But somehow, we need a user-friendly method for users to run software on their computers.

Resources

One structure that I want to exist that doesn't currently fit in is a "static resource". The goal of a static resource is to take advantage of the requests bursts combined with network topology. It is likely that for certain resources (such as a viral image or a current broadcast) many people will be requesting it in a short period of time. We want to provide a means for nodes to hold on to that resource and fulfill the request. The only caveat is that they have to follow the rules and pay the resource owner appropriately. Some form of double book-keeping would work for this.

Notes

Rent Control

Just an idea, but it may work. The problem is that I want there to be a lot of competition within DA servers and RID servers. But they will commodity, which guarantees a race to the bottom. Allow applications (repos) to specify a minimum price that DA servers can charge. If we keep this value a bit above the cost of cloud infrastructure, than anyone (with a bit of a technical background) can run a server. This should guarantee more competition. It should also reduce (though only slightly) the risk of dishonest DA servers.