Data

Data; it's structure, formatting and storage.

Large Uint

A uint is a byte slice.

 (data uint) LoadUint(l uint) (this uint, rest uint) {
   if l==0{
     b = data[0]
     if b < 129 {
       return b, data[1:]
     }
     if b < 252 {
       return data[1:b-250], data[b-250:]
     }
     return LoadUint(b-251)
   }
   return data[1:l+1], data[l+1:]
 }

Fundamental Structures

Value

A type and bytes where the type indicates how to interpret the bytes.

Anonymous Type

The type is part of the value.

List

An ordered list, all of the same type. Variable length.

Tuple

An ordered list where the tuple type defines the length and the type of each element. Last variadic?

Dict

An unordered list where each element has key and value.

Struct

A set of fields. This also necessitates a field as type. A field is a tag (field name) and a type.

Tagged Union

One of a set of types with a tag indicating the type.

I was considering an enum, but the List may be sufficient.

Mono Container

I think I can represent all these types with a single container.

• Key Type
  • Ordered: The key is an int taken from the position, an ordered list
  • Type: The keys all have the same type.
• Key Value
  • Direct : The data will define the key value directly
  • Indirect : The data will define a location to get the key value.
• Value Type
  • None : The data will define the value type
  • Lookup : The combination of ContainerType, KeyType and KeyValue provides a lookup - not sure how to make this work.
  • Type : All the data has the same type.

The language is completely regular. No parser is needed. I'm not sure if it should be stack based, register based or both. But the size must be defined in the script.

The language also is not Turing complete. No loops and no conditionals.

Shared Context

An important feature is the ability to read data from a shared context. But this requires collision avoidance so the keys in a shared context are large. We don't want these large values repeated in the data, so there is an operation to read from the shared context into a register.

IDs

IDs are either set, which allows for references to be built up or are derived from hashes.

Quick Lookup and Compact Data

When data is going to be fully deserialized, compaction tends to be advantageous. But for data at rest this causes lookups to be slow. If a struct has 5 variable length fields and we want the fifth, we have to pass through the first four.

To make a struct in place that allows for fast lookups, we need a constant header size. Lengths need to be a constant size. But even defining lengths will still require reading all four, so instead we encode offsets. For example if we have

 Person:
   ID: int64;
   Age: byte;
   RoleID: byte;
   First: string;
   Last: string;

Then the first 3 fields are fixed length. We have to place an upper limit on the length of string. So lets say we allocate 4 bytes for this. Then we now First starts at byte (8+1+1+4+4) 18. The 4 byte value in the First header field will actually be the relative offset of the end of first. And Last will be the relative offset of the end of Last as well as the end of the record.

This further makes an argument to place the variable length field headers at the top, in reverse order. So the very first piece of data is the record length.

Namespace

A structured namespace can be defined. A secure model would be something like

• Resource (url)
• Public Key
• Hash
• Hash bits

Then the ID would be the defined bits of the hash. This would allow for fewer bits to be used to identify static resources.