Every Database Had Its Own Storage Engine — Then Came Substrate Architecture.

From Vertically Integrated Engines to Reusable Storage Substrates

Database architecture has repeatedly moved toward greater separation between what a database means and how its state is stored and managed. Early systems tightly coupled application semantics to storage mechanisms; later systems introduced general-purpose database engines, specialized data models, and increasingly reusable infrastructure beneath them.

The latest step is a shift from treating the database engine as a monolithic system toward treating it as a composition of distinct responsibilities. Persistent storage, transactions, concurrency control, recovery, and replication can increasingly be provided by a reusable substrate, allowing a higher layer to define the semantics that make one database system different from another.

This article traces that progression through four generations of database architecture and introduces TaxisDB as a particularly clear example of the emerging separation between a reusable state-management substrate and the database semantics layered above it.

Four Architectural Generations

The history of database architecture can be understood as a gradual movement of abstraction boundaries. Each generation changes not only how data is stored, but also the logical model programmers use to reason about data and which parts of the system can be independently reused.

Gen.1 (1960–1979) — Physical Databases

In the first generation, spanning roughly 1960–1979, the database engine was fundamentally a physical construct. The programmer thought in terms of files, records, pointers, record layouts, pages, and access paths. Data structures and storage structures were closely intertwined with application logic. There was little conceptual distance between the logical representation of data and the mechanisms used to store and retrieve it.

This approach could be extremely efficient, but it made applications tightly coupled to their storage mechanisms. There was no strong abstraction boundary protecting the programmer from the details of storage.

Gen.2 (1979-2009) — Relational Abstraction

The second generation, spanning roughly 1979–2009, introduced a radically different abstraction: the relational model. Codd’s contribution was the separation of data’s logical meaning from the mechanisms used to store and retrieve it. The programmer could reason in terms of relations, tuples, predicates, and declarative queries rather than files, pointers, or physical access paths.

This separation allowed the DBMS to determine how a logical request should be physically executed. The same query could be evaluated using different indexes, join algorithms, access paths, or physical layouts without requiring the application to change. Physical implementation became an internal concern of the DBMS rather than part of the application’s programming model.

The RDBMS Owns the Stack

A traditional SQL (relational) database management system (RDBMS) owns essentially the whole stack, from SQL down to the physical disk:

flowchart TD

    A[SQL] --> B["Relational model<br/>tables / tuples / joins"]

    B --> C[Query optimizer]

    C --> D["Index structures<br/>B-trees / hashes"]

    D --> E["Storage engine<br/>pages / WAL / MVCC"]

    E --> F[Files / filesystem]    

PostgreSQL, for example, owns the semantics of tables, tuples, transactions, indexes, MVCC, WAL, recovery, storage pages, buffer management, and replication mechanisms. The relational abstraction is packaged together with a substantial amount of storage machinery. That is what “owning the storage stack” means.

Where the Relational Abstraction Stops

The 1979–2009 era was dominated by vertically integrated relational database systems, in which the logical relational model and the physical storage machinery were encapsulated within a single RDBMS product.

What became limiting was not the relational abstraction itself, but where the abstraction boundary was placed. The separation between logical data and physical implementation was established inside each RDBMS, rather than between independently reusable architectural layers.

The RDBMS could hide its physical implementation from the programmer, but the storage machinery remained part of the RDBMS itself. PostgreSQL has a clear abstraction between its relational model and its physical implementation, but PostgreSQL still owns the underlying storage stack. MySQL owns its own storage machinery. Oracle owns its own. Microsoft SQL Server similarly maintains its own storage layer.

The result is that the physical layer is abstracted, but not independently reusable. Each RDBMS maintains its own implementation of storage, transactions, concurrency, recovery, and related machinery:

flowchart LR

    MySQL[MySQL] --> M1[its own storage machinery]

    Oracle[Oracle] --> O1[its own storage machinery]

    SQLServer[SQL Server] --> S1[its own storage machinery]

The relational revolution therefore solved one problem while leaving another largely untouched. It separated the logical model from the physical implementation, but it did not separate the RDBMS from the storage substrate.

Gen.3 (2009-2019) — NoSQL Movement

The third generation, spanning roughly 2009–2019, was defined by the rise of the NoSQL movement. It emerged in response to the new demands of internet-scale systems: massive datasets, high request volumes, horizontal scaling, high write throughput, continuous availability, and workloads that did not fit naturally into relational tables and joins.

The movement challenged an important assumption of the relational era: that a general-purpose relational data model and its associated access patterns were the appropriate abstraction for every workload. Instead, NoSQL systems introduced data models and access patterns designed around particular classes of applications. The result was another major shift in logical abstraction, but not yet a fundamental change in how database systems were architected.

What NoSQL Actually Changed

NoSQL systems challenged the assumption that every workload should be represented as relations and accessed primarily through joins and declarative relational queries. MongoDB introduced a document model, Neo4j a property graph, Cassandra a wide-column model, and Redis a key-value model.

flowchart LR

    MongoDB[MongoDB] --> Documents[Documents]

    Neo4j[Neo4j] --> Graph["Property graph"]

    Cassandra[Cassandra] --> WideColumn["Wide-column"]

    Redis[Redis] --> KV["Key / Value"]

This was a genuine logical and data-model revolution. The programmer could now choose an abstraction that matched the structure and access patterns of the workload rather than forcing every application into the relational model.

But underneath the logical model, the architecture changed significantly to support the requirements of each new data model and workload. Each system developed specialized machinery to turn its logical abstraction into durable, concurrent, recoverable, and distributed state, while continuing to own that machinery as part of the DBMS itself.

The Database Engine Remained the Unit of Packaging

A NoSQL system therefore did not generally separate its logical model from its physical infrastructure in a new architectural sense. Instead, it built a complete database engine around a different logical model.

flowchart TD

    subgraph M[MongoDB]
        M1[Document model]
        M2[Query & execution]
        M3[Indexes]
        M4[Transactions]
        M5[Replication]
        M6[Sharding]
        M7[Storage engine]
        M8[Persistence & recovery]
    end

    subgraph N[Neo4j]
        N1[Graph model]
        N2[Cypher & execution]
        N3[Graph indexes]
        N4[Transactions]
        N5[Clustering & replication]
        N6[Storage engine]
        N7[Persistence & recovery]
    end

Structurally, the pattern remained familiar: build a complete DBMS, but optimize its architecture around a different logical model and workload. These systems were solving genuine problems around internet-scale traffic, massive datasets, horizontal scaling, high write throughput, availability, and workload-specific access patterns. Tight integration between the logical model and physical storage could be a significant advantage. NoSQL changed the abstraction presented to the programmer and, in many cases, radically changed the distributed architecture underneath it. Yet each DBMS still generally owned its indexes, transaction machinery, replication, recovery, and storage layer.

The important point is architectural: the industry continued to treat the DBMS as the unit of packaging. The NoSQL movement diversified the logical layer and reshaped the architecture underneath it, but it did not yet make the physical layer a generally reusable substrate.

Gen. 4 (2019–Today) — Reusable Storage Substrate

Two Problems Instead of One

The fourth generation begins with a different architectural decision. Building a new DBMS, such as a temporal graph database or a distributed database engine, historically meant solving two problems at once. The first was the database itself: its semantics, data model, query language, indexes, query planning, and constraints. The second was the infrastructure beneath it: durable storage, replication, distributed transactions, failure recovery, concurrency, partitioning, consistency, node-failure handling, and cluster reconfiguration.

That is a substantial amount of engineering to reproduce for every new database system. A storage substrate changes the equation by providing a reusable computational foundation on which another database abstraction can be constructed. The substrate provides general mechanisms for managing persistent and distributed state while leaving higher layers free to define their own data models and semantics.

From Abstraction to Factoring

This extends the abstraction introduced by the relational model. Codd’s abstraction says, in effect, do not expose the physical mechanism to the logical user. The substrate idea goes one step further: Do not necessarily require every logical database system to implement its own physical state-management machinery.

This is the crux of the distinction. Codd’s relational model exemplified abstraction; substrate architecture extends that idea into architectural factoring, making underlying database infrastructure independently reusable across multiple logical database systems. The first separates an interface from its implementation. The second turns that implementation boundary into a reusable architectural component that can support multiple database abstractions.

Codd’s relational abstraction separates logical meaning from physical representation. Modern substrate architectures seek to separate database semantics from the machinery responsible for managing persistent and, where applicable, distributed state.

Why Build the Same Infrastructure Again?

The substrate approach becomes attractive when multiple database models require similar storage and state-management primitives. The NoSQL movement argued that one data model does not fit every workload.

Examples of Rich Independently-Reusable Substrate Architecture

The examples that follow illustrate a stronger form of substrate architecture. They differ in where the boundary between the substrate and the higher-level database is drawn and in the capabilities exposed through that boundary, but share the same fundamental property: substantial persistence and state-management machinery is provided by an independently reusable system rather than implemented anew within the higher-level database.

• Kronotop is a distributed, transactional document database backed by FoundationDB for consistency, coordination, and fault tolerance. It provides two data models: Bucket (documents, with secondary indexes, vector search, and a query language) and ZMap (ordered key-value) behind a shared RESP interface, transaction model, and namespace system. FoundationDB supplies distributed persistence, strict serializable transactions, conflict detection, sharding coordination, and cluster state, storing all metadata, indexes, and ZMap data directly. Document bodies, which exceed FoundationDB’s value-size limits, are instead stored on local disk by Volume, Kronotop’s own storage engine, with only pointers kept in FoundationDB.

• FoundationDB Record Layer uses FoundationDB as a distributed, transactional, strongly consistent storage substrate. It implements a record-oriented structured data model using Protobuf-defined records, supporting nested and hierarchical schemas. The Record Layer provides schema management, secondary indexes, declarative queries, and query planning on top of FoundationDB’s ordered key–value abstraction. FoundationDB supplies distributed persistence, replication, fault tolerance, concurrency control, and strict serializable transactions, allowing the Record Layer to focus on higher-level database semantics within a stateless, multi-tenant architecture.

• Octo is a relational SQL query engine built on top of YottaDB, whose native data model is based on hierarchical, multidimensional sparse arrays known as globals. YottaDB provides persistence, transactions, concurrency control, journaling, recovery, and replication capabilities. Octo adds a relational layer over this existing database model, including tables, schemas, indexes, relational algebra, SQL parsing, query planning, and execution. Relational structures are represented within YottaDB globals, making YottaDB a substantially richer substrate than a conventional storage engine, while also creating a tighter coupling between Octo’s relational implementation and YottaDB’s underlying hierarchical data model.

FoundationDB provides distributed state-management machinery through a relatively general ordered key–value abstraction. YottaDB provides transactional and replication machinery through a multidimensional, hierarchical, sparse key–value data model and an in-process, daemonless architecture. They therefore demonstrate different forms of substrate reuse based on fundamentally different underlying storage models.

The New Architectural Question Revisited

With reusable substrates, the question “Is this a SQL database or a NoSQL database?” becomes less fundamental.

Once storage and state management are factored into a reusable substrate, a second question becomes possible:

The examples above illustrate different answers to this question:

Multi-Model Databases

From another perspective, the architectural separation between database semantics and underlying storage can also be examined through multi-model databases. Rather than building on a rich, independently reusable substrate of the kind described above, these systems typically place a reusable storage engine or pluggable storage service beneath the database logical layer.

Examples of Multi-Model Databases

• TypeDB uses RocksDB as an embedded storage engine. Its fundamental data model is a schema-first polymorphic entity–relation–attribute (PERA) model, represented as a typed hypergraph. TypeQL is its modeling and query language, providing declarative schema definition, pattern matching, inference rules, and reasoning. TypeDB implements its typed data model, transaction semantics, inference, indexing, and query execution above RocksDB.

• Datomic uses pluggable storage services, including DynamoDB, Cassandra, and SQL databases such as PostgreSQL. Its fundamental data model is the immutable datom-based EAVT model, with Datalog as its query language. Datomic implements the immutable and temporal database model, indexing, query processing, and logical transaction semantics above the storage layer, while the selected storage service provides the underlying persistence.

• SurrealDB is a multi-model database supporting document, graph, relational, time-series, vector, and geospatial data, with SurrealQL as its primary query language and GraphQL support. Its fundamental data model is record-oriented, with records providing the underlying representation for its higher-level models. Its architecture separates the logical database and query layers from a pluggable key-value storage subsystem, with RocksDB, SurrealKV, or TiKV for single-node and self-managed multi-node deployments, and SurrealDS, its own distributed engine built on object storage, for horizontally scaled and managed deployments.

• ArcadeDB is a multi-model database supporting graph, document, key-value, search, vector, and time-series data, with SQL and Gremlin as its primary query languages. Its fundamental data model is record-oriented, with vertices, edges, and documents represented as records within a common storage model. ArcadeDB provides the higher-level data models, indexing, and query processing within a native multi-model engine that shares storage and transaction infrastructure across the different models.

The Common Key–Value Representation

Despite their architectural differences, these systems share an important characteristic with FoundationDB and YottaDB: their underlying storage can be understood in terms of a key–value representation. FoundationDB exposes an ordered key–value model directly, while YottaDB organizes data as multidimensional, hierarchical sparse key–value structures. TypeDB, Datomic, SurrealDB, and ArcadeDB likewise ultimately represent their data through key–value-oriented storage structures, although the abstraction exposed to the database layer and the semantics built above it differ substantially. The commonality is therefore not that they provide the same storage architecture, but that key–value representation serves as a fundamental building block beneath higher-level database models.

Effects of the Fundamental Data Model on Multi-Model System Architecture

The systems above differ substantially in the fundamental abstraction from which their multiple models are constructed:

• TypeDB is built around entities, relations, and attributes, with a strong type system. Graph structure is represented through typed relations and named roles, including n-ary relations and relations that can themselves participate in other relations.

• Datomic is built around the immutable datom, an entity–attribute–value fact associated with a transaction. Graph structure is represented through reference-valued attributes, where datoms connect entities to other entities; these references can be traversed in both directions.

• ArcadeDB takes a record-oriented approach in which everything is a record. Its graph model is a property graph, with vertices and edges as first-class records; edges connect vertex records and can themselves carry properties.

• SurrealDB likewise uses records as a fundamental abstraction. Its graph model supports both record links, where a record directly references another record, and graph relations, where edges are represented as separate records with in and out references. This allows relationships themselves to carry properties while remaining part of the record-oriented model.

Physical representation

The fundamental data model determines how logical objects are encoded into persistent state. Entities, facts, records, and relationships may be stored as distinct physical structures, or encoded into keys, values, indexes, and metadata. Even when systems share a similar underlying key-value substrate, the model dictates what gets encoded and how it is organized.

Schema evolution and instance creation

Some systems require entity types, attributes, and relationships to be declared before instances can exist, and require explicit changes to those declarations as the schema evolves; others let records acquire and change structure dynamically through their own fields and values. This determines how a type is defined, how an instance is created, and how the database checks whether a stored instance conforms to that type. As the schema evolves, it also determines how existing instances are interpreted and whether instances created under different schema versions can coexist.

Indexing and identity

Indexes must correspond to the objects and access patterns the model defines. Unique-key lookup depends on how identity itself is represented: what makes an instance uniquely identifiable in the first place.

Constraints, rules, and reasoning

Constraints, rules, and inference all operate on the model’s fundamental abstractions. Constraints may attach to entity types, attributes, relationships, or record fields. Rules may derive new relationships or facts, validate structural conditions, or enforce invariants over records. Where reasoning is supported, the model determines what can be inferred and how those inferences relate to the stored data.

Relationships

Relationships may be first-class objects, references embedded in records, separate edge records, or facts connecting entities. This determines whether a relationship can carry its own identity, attributes, type, and relationships with other objects.

Composite Values

Composite values raise a related question: whether a structured value stays embedded within its containing entity or record, or becomes a separately identifiable object with its own type, identity, and relationships.

Insert and upsert semantics

Insert and upsert behavior depends on what the database treats as an instance and how it establishes identity: creating immutable facts, mutating an identified record, merging on a unique key, or creating a new entity when no matching identity is found.

Summary

The fundamental data model therefore sets off a chain of architectural consequences: it shapes the physical representation of state, the schema and type system exposed to developers, the representation and identity of instances and relationships, and the semantics of indexing, constraints, rules, reasoning, evolution, and data modification.

TaxisDB: Datomic-Style Database on YottaDB and a Freebase-Style Schema on Top of It

The preceding examples illustrate how both the choice of substrate and the fundamental data model shape the architecture of a database system. TaxisDB brings these two dimensions together in a particularly distinctive way: it uses YottaDB as a reusable state-management substrate, adopts an immutable, fact-oriented data model inspired by Datomic, and provides a Freebase-style reusable schema through TaxisBase.

Octo’s relationship to YottaDB already provided one example of this architectural separation, with YottaDB carrying the underlying state-management responsibilities while the logical layer defines its own database model. TaxisDB reuses the same substrate, but applies it to a fundamentally different data model.

TaxisDB is a metamodel-driven, reflective, temporal, immutable, fact-oriented database engine implemented on top of YottaDB. Where Octo layers a relational model over YottaDB’s globals, TaxisDB uses an immutable fact model in which facts, rather than rows, are the fundamental unit and the schema itself is represented as data that the database can inspect and reason about.

The Division of Labor

TaxisDB                              YottaDB
 ├── Facts                            ├── Persistent hierarchical storage
 ├── Immutability                     ├── Ordered sparse n-dimensional arrays
 ├── Temporal semantics               ├── Transactions & concurrency control
 ├── Identity                         ├── Journaling & recovery
 ├── Metamodel / reflection           └── Replication
 ├── Indexing
 └── Query & database APIs

TaxisDB owns indexing, consistency rules, and query semantics, while YottaDB’s native hierarchical model inevitably shapes how TaxisDB lays out its physical representation, the same coupling already noted for Octo. What TaxisDB illustrates, more sharply than Octo, is how far the logical layer can diverge from anything relational while still sitting on the same Type A substrate: reusable transactional infrastructure plus a new logical model equals a new database system, with the substrate handling durable state so the database layer can concentrate on what makes it distinct.

TaxisDB owns the semantics that make it a distinct database system. In particular, it provides a semantic vocabulary for entities, attributes, values, domains, ranges, and value dictionaries; write-time constraint enforcement; multiple forms of entity identity; immutable assertions and retractions with historical context; and a two-phase Stage → Transact write path. The Stage phase performs validation and identity resolution without modifying durable state. The Transact phase commits the resulting changes atomically to YottaDB. This keeps storage interaction concentrated in a small, transactional boundary.

Why YottaDB?

The motivation is not simply to use an unusual storage engine. It is to test the Gen. 4 proposition that a mature transactional substrate can support a substantially different database abstraction.

Datomic demonstrated the value of separating immutable-fact semantics from the underlying storage service. TaxisDB applies a similar architectural principle to YottaDB, with an embedded transactional substrate rather than a cloud-oriented storage architecture.

YottaDB brings decades of production history, transactional persistence, journaling, recovery, and hierarchical storage. Most important YottaDB provides multi‑process ACID database design without a server. TaxisDB does not need to reproduce those mechanisms; it can concentrate on the semantic layer that distinguishes the system.

Query as a Higher Level Concern

TaxisDB deliberately does not define its own query language. Its core responsibility is the definition, revision, control, identity, and history of database state, while querying can be provided at a higher level in different forms.

There is a useful conceptual correspondence with the traditional SQL categories Data Definition Language (DDL), Data Manipulation Language (DML), Data Control Language (DCL), Transaction Control Language (TCL), and Data Query Language (DQL), although TaxisDB does not and will not attempt to reproduce SQL as a language.

TaxisDB already provides capabilities corresponding to parts of these subsystems. DDL-like operations define and evolve the database model, including types, attributes, domains, ranges, and other schema structures. DML-like operations create and retract facts and manage entities and their relationships. TCL-like behavior is provided through TaxisDB’s transactional write path, with Stage → Transact separating preparation from atomic durable commitment. DCL-like concerns can be represented through the database’s control and constraint mechanisms

DQL is intentionally left outside the core. A higher-level query layer can provide the appropriate read language for a particular application or data model. Possible interfaces include Datalog, GraphQL, openCypher, or other declarative and graph-oriented query languages. Multiple query interfaces could therefore operate over the same underlying TaxisDB semantics without requiring the core engine to adopt any one query language.

Currently, TaxisDB uses YDB Octo as its SQL query interface. Octo exposes SQL over the YottaDB globals in which TaxisDB’s data is stored, providing a practical way to retrieve and inspect the database state without making SQL query planning part of the TaxisDB core.

The resulting separation can be summarized as:

graph TD
  subgraph Q[Higher-level query layer]
    Q1[DQL → Datalog / GraphQL / openCypher / Octo SQL / ...]
  end

  subgraph X[TaxisDB]
    X1[DDL-like → model definition]
    X2[DML-like → facts and entities]
    X3[TCL-like → transactional state changes]
    X4[DCL-like → control and constraints]
    X5[identity + history + temporal semantics]
  end

  subgraph Y[YottaDB]
    Y1[persistence + transactions + concurrency + recovery + replication]
  end

  Q --> X --> Y

This is therefore an architectural choice rather than an unfinished feature: TaxisDB defines and manages database state, YottaDB provides the underlying storage and transactional substrate, and higher-level query systems provide the read/query interface.

Separating the Model from the Data

A fundamental principle of TaxisDB is the separation between entity types that describe a domain and concrete instances that populate the domain. In knowledge representation literature, this distinction is commonly expressed as the separation of the TBox and ABox.

• The TBox describes the model. It defines the kinds of things that can exist, their properties, relationships, constraints, and meaning. It provides the vocabulary and structure used to describe a domain.

• The ABox contains the concrete assertions expressed using that model. It describes actual entities and the facts that hold about them.

For example, a model may define:

• a Person entity type
• a Gender value type
• a Geolocation value type
• properties that describe people and their relationships

Those definitions belong to the TBox. Concrete instances such as a particular person, a particular gender value, or a particular geographic location belong to the ABox.

This creates a clear distinction between:

• Entity Types — definitions of things such as Person, Gender, and Geolocation
• Entities — concrete instances such as Alice, male, and (-36.60664, -72.10344)

An Entity Type is the blueprint; an Entity is something created according to that blueprint.

This separation is important because the model and the data have different roles and different lifecycles. The model establishes the vocabulary and rules within which data is interpreted, while the data represents the facts expressed using that vocabulary. Keeping these concerns distinct makes the system easier to reason about, inspect, evolve, and reuse, while also providing clear boundaries for security, performance, governance, and data sharing. Model definitions can be managed and protected independently from domain data, while data can be optimized, partitioned, shared, or exchanged without necessarily changing the definitions that give it meaning.

At the same time, the separation does not mean that the model is external to the database. TaxisDB treats the model itself as first-class data. The definitions that describe the domain can therefore be represented, inspected, and evolved within the same overall data model as the facts they describe.

This gives TaxisDB a layered semantic structure:

Model
  ↓
Entity Types
  ↓
Entities
  ↓
Facts about those entities

The distinction between definitions and instances is therefore the foundation on which TaxisBase can introduce a reusable vocabulary. TaxisBase can define general-purpose semantic patterns at the model level, while application-specific schemas can use those patterns to describe their actual domain data.

TaxisBase: A General-Purpose Schema Above TaxisDB

TaxisDB provides a minimal sys namespace containing the bootstrap concepts required to define further models. TaxisBase is intended to build a broader, reusable vocabulary on top of that foundation. It is not a separate database engine or storage layer; it is a general-purpose schema and metamodel built using TaxisDB’s own primitives.

Schema as Data

A fundamental property of TaxisDB is that the database model itself is represented as first-class data. Rather than treating schema primarily as external database configuration, TaxisDB represents concepts such as types, attributes, relationships, constraints, and other model structures within its own fact model.

This provides the foundation on which TaxisBase is built. TaxisBase does not require a separate schema mechanism; it uses the same TaxisDB primitives to define and represent its vocabulary.

Schema-as-data enables several important capabilities:

• Runtime schema discovery — applications can inspect the available types, attributes, relationships, and other model structures at runtime.
• Dynamic model evolution — model definitions can themselves be introduced or changed through database operations, subject to the constraints of the model.
• Reflection over database structures — the database can inspect and operate on representations of its own model.

This creates a recursive relationship between the database and its model: the database stores domain facts, but it can also store facts describing the structures used to interpret those facts.

Its design is influenced by Freebase, particularly two ideas: entities can participate in multiple typed facets, and many real-world facts cannot be adequately represented as a simple entity/attribute/value assertion. TaxisBase addresses the first through Objects and Associations. An Object has a single foundational type, preserving a simple and unambiguous core identity. Additional characteristics are represented through Associations, which attach distinct typed facets to the same Object, so an entity can participate in multiple conceptual roles. The second idea is addressed through Composite Value Types (CVTs), which represent a fact whose meaning depends on multiple related values rather than a single scalar value.

Reusable Semantic Vocabulary

TaxisBase is therefore intended as a reusable semantic vocabulary above TaxisDB, providing common modeling patterns that application-specific namespaces can build upon. Instead of each application independently inventing mechanisms for multi-faceted entities, n-ary relationships, and composite facts, those patterns can be defined once in TaxisBase and reused across domains.

graph TD
  A[Application schemas] --> B

  subgraph B[TaxisBase]
    B1[reusable vocabulary]
    B2[Objects / Associations]
    B3[Composite Value Types]
  end

  B --> C

  subgraph C[TaxisDB]
    C1[fact model / identity]
    C2[temporal history / schema-as-data]
    C3[write & transaction semantics]
  end

  C --> D

  subgraph D[YottaDB]
    D1[transactional storage substrate]
  end

The important architectural distinction is that TaxisBase adds reusable semantics, not another storage or execution layer. TaxisDB provides the mechanisms for representing and managing facts, history, identity, and the model itself as data; TaxisBase uses those mechanisms to define a higher-level vocabulary for expressing richer domain structures.

In this sense, TaxisBase is the first reusable semantic layer above the core TaxisDB metamodel: application schemas can build on a common vocabulary rather than repeatedly defining their own mechanisms for modeling complex facts.

Attribute Dictionary Encoding

TaxisDB currently uses per-attribute dictionary encoding to separate logical attribute values from their physical representation in the immutable fact indexes. Rather than repeatedly storing arbitrary strings, numbers, or other values, each attribute maintains an ordered value dictionary in YottaDB that maps logical values to compact, attribute-local keys.

The dictionary is attribute-specific, allowing the physical representation to be associated with the value domain of each attribute. The fact is represented in the EATV global as:

^EATV(eid, aid, tx, valkey) = op

where valkey identifies the value through the dictionary associated with that attribute. The implemented mapping is therefore:

attribute + value ↔ valkey

The EATV ordering is:

eid → aid → tx → valkey

This ordering is particularly suited to TaxisDB’s immutable temporal model. A datom is an immutable historical statement about the truth state of a fact at a particular transaction. Because datoms for an entity are physically clustered by eid, and datoms for an attribute are ordered by tx, the complete history of an entity can be traversed directly from the primary EATV index. The current state is derived by evaluating the latest datoms for each (eid, aid) pair. EATV therefore makes entity reconstruction and temporal state resolution a natural physical access pattern.

The attribute dictionary reduces the physical cost of this representation. Repeated values for an attribute share the same valkey, so frequently occurring values do not have to be repeatedly stored in the immutable indexes. This increases index density and can improve cache locality, storage efficiency, and value comparison costs.

The dictionary keys can preserve the ordering of their logical values:

value₁ < value₂  ⇒  valkey₁ < valkey₂

When this ordering is preserved, dictionary encoding is more than a compression mechanism: ordered operations such as range scans and inequality predicates can operate on compact keys while retaining the logical ordering of the underlying values.

The attribute dictionary also provides a natural point for type and value-range enforcement. A value can be validated against the attribute’s domain and resolved to its valkey during the Stage phase, before the resulting fact reaches the transactional commit boundary.

A further extension is being considered but has not yet been implemented. The current attribute-local mapping could be extended with a second mapping from dictionary entries to canonical literal values:

fact
  ↓
attribute dictionary / valkey
  ↓
canonical literal

Under this design, each literal could be stored once and referenced wherever it occurs. Common integers, strings, words, floating-point values, and other canonical values could therefore be shared beyond an individual attribute dictionary. This could further reduce physical duplication and has potentially important implications for merging independently produced datasets, since equivalent literals could be resolved to a common canonical representation rather than duplicated during a merge.

The current implementation therefore provides:

fact → attribute dictionary → value

while the proposed extension would provide:

fact → attribute dictionary → canonical literal

The distinction separates three concerns: the datom records the assertion and its temporal truth state; the attribute dictionary provides compact, attribute-specific value identity; and the proposed literal layer would provide shared canonical value identity. The first is implemented today; the second is a future extension.

Epilogue: TaxisDB and the Question of Layered Responsibility

The four generations discussed in this article reflect a continuing shift in where database responsibility is placed, between the semantics that define a database system and the infrastructure it can reuse. TaxisDB demonstrates an emerging architectural direction in which this boundary is placed unusually high in the stack, separating database semantics from the machinery responsible for managing persistent state.

The resulting architecture is a three-layer separation of responsibility. At the foundation, YottaDB provides the state-management substrate: persistence, transactions, concurrency, recovery, journaling, and replication. On top of it, TaxisDB defines the database semantics: immutable facts, identity, history, metamodel and schema-as-data, constraints, and transactional state changes. Above TaxisDB, query and retrieval form the third layer, allowing different query models to operate over the same underlying database state.

This separation gives TaxisDB its distinctive character. Its significance lies not in any single mechanism, but in their composition: a serverless transactional substrate, an immutable temporal fact model, a reflective metamodel, attribute-local value representation, and deliberately decoupled query and retrieval mechanisms.

At the foundation, YottaDB’s serverless architecture allows TaxisDB to inherit decades of battle-tested multi-process ACID engineering, with multiple independent OS processes acting as peers that directly access the same database files without a daemon or network hop. Concurrency control, journaling, recovery, isolation, and durability are coordinated at the process level through shared memory and locking, providing a mature transactional foundation without the complexity of distributed consensus. Most importantly Its hierarchical globals remain directly inspectable and manageable through the YottaDB environment, giving TaxisDB an unusual degree of operational transparency and making the underlying storage model readily accessible for experimentation..

Within the TaxisDB layer, the EATV primary index provides a physical organization suited to temporal entity reconstruction, while attribute dictionary encoding separates logical values from their physical representation in immutable indexes. TaxisDB also separates the model from the data, distinguishing the definitions that describe a domain from the concrete entities and facts that populate it. TaxisBase builds on this separation and the reflective, schema-as-data model to provide a reusable semantic vocabulary for richer domain structures.

At the third layer, query and retrieval form a distinct higher-level concern. TaxisDB does not define a query language as part of its core semantics; instead, query mechanisms such as Octo SQL, Datalog, GraphQL, or openCypher can operate over the database state managed by TaxisDB.

This combination places TaxisDB in an unusual position among existing database architectures. It is not simply a graph database, because its fundamental abstraction is the immutable temporal fact rather than graph traversal. It is not simply an EAV database, because its metamodel, identity, history, constraints, and temporal semantics form a more complete database model.

And while it borrows important ideas from fact-oriented systems such as Datomic, its separation of the model from the data, EATV organization, attribute dictionaries, reflective schema architecture, and direct use of YottaDB’s serverless hierarchical storage give it a distinct physical and architectural character.

Its distinctive position instead comes from the composition:

graph TD
    A["Query / Retrieval<br/>Octo / Datalog / GraphQL / ..."]
    B["TaxisDB + TaxisBase<br/><br/>model · entity types · entities<br/>facts · immutable history · identity<br/>constraints · schema-as-data<br/>EATV · value dictionaries<br/>reusable semantic vocabulary"]
    C["YottaDB<br/><br/>serverless persistence · ACID · concurrency<br/>recovery · journaling · replication<br/>hierarchical globals"]

    A --> B
    B --> C

The architectural proposition is therefore broader than simply building another database on top of YottaDB. A mature transactional substrate can support a substantially different database abstraction while allowing its physical representation to remain inspectable and operationally accessible; its fact model to remain immutable and temporal; its schema to describe itself; its value representation to be separated from assertions; reusable semantics to be layered above the core; and query languages to remain independently composable.

Its answer is provocative. The distinctive identity of a database may lie not in owning every layer of the stack, but in defining a new set of semantics on top of infrastructure that is already capable of managing durable state.

The architectural proposition of TaxisDB is therefore the deliberate separation of three responsibilities: TaxisDB occupies a middle layer that connects to the substrate to manage durable state through the data access layer, and connects to the query engine to retrieve data. The substrate does not need to understand those semantics, and retrieval mechanisms do not need to define them; they connect through the database core that gives the underlying state its meaning.

The database, in this architecture, is the semantic core that connects the substrate to the ways data is retrieved.