openmdta.

Under the hood / Technical Overview

The platform behind your next product.

The core infrastructure is implemented: durable ingestion, background recovery, structured data, search, calculated key figures, latest values, tick history, OHLC candles, corporate-action consolidation, split adjustment, durable observation tasks, and typed SDKs. We build on that foundation to implement the provider-specific parsers and integrations your product needs.

Evidence reviewed 15 September 2026. Early customer evaluation; no customer production references are published here.

01 / THE DATA MODEL

A Dataset defines what you cover.

Which entities do you have information about, and what can your product do with it? A Dataset brings together a Catalog and, optionally, one canonical Stream for the entities it covers. The Catalog describes those entries; the Stream carries their time-ordered updates.

Each Dataset has its own coverage: a broad instrument universe, a venue’s listings, or a specialist set of structured-product terms. Coverage is actual membership—not a category label, and not a promise that every field or historical period is populated.

Catalog — Model the things your product needs to understand.

A Catalog holds structured records: instruments, listings, issuers, financial observations, classifications, and relationships. Define typed fields, identifiers, and dimensions around your data. Combine provider information with your own fields while retaining provenance and access requirements.

  • Instruments and listings: identifiers, names, currencies, venues, and product terms.
  • Companies and issuers: legal entities, classifications, and issuer relationships.
  • Fundamentals and estimates: observations organized by subject, metric, period, and provider.
  • Analyst data: structured ratings, price targets, and recommendations.
  • Funds: holdings and constituent records linking funds to instruments.
  • Corporate actions: dated events, split ratios, statuses, and distributions.

A Catalog is not a fixed instrument schema or a static file. Its records change as information arrives; your schema defines what those records mean.

Stream — Model what happens over time.

A Stream carries a source-ordered sequence of typed updates: quotes, trades, or other changing values. Your schema defines the payload. OpenMDTA supplies live delivery, retained history, explicit gaps, and recovery around it.

Source positions make continuity explicit. Live processing continues alongside background recovery; recovered history does not overwrite newer live state. Anything still missing remains visible.

For example, a Catalog describes an instrument and its listings. A Stream carries quote and trade updates. Services expose the current values, individual historical ticks, and aggregated candles.

The distinction is how you model and consume information—not simply “slow data” versus “fast data.” A Catalog-only Dataset can provide specialist information without a live Stream.

Data coverage and financial logic

These are modeling examples, not bundled datasets. Provider integration, reporting periods, revisions, and financial semantics need defining for your use case. Holding ETF constituent data is not the same as implementing complete recursive look-through calculations.

News ingestion, archiving, and delivery are outside the current core offering.

OpenMDTA supplies a platform for organizing, calculating, searching, and distributing data—not a bundled financial-data subscription or a complete Terminal replacement. Coverage comes from the data you own or license and the integrations we implement together.

02 / CONNECT THE IDENTIFIERS

Different identifiers. The same entity.

One provider uses an ISIN, another a WKN, a third its own identifier. Connect their information through shared identity resolution instead of rebuilding cross-provider mapping tables in every application.

Typed identifiers. Traceable connections.

Identifiers carry their type: ISIN(…), WKN(…), CUSIP(…), or a configured vendor-specific type—not an unexplained string. Source records assert which identifiers refer to the same entity. Identity resolution connects those claims within the appropriate entity type and selects a leading identifier using configured priorities.

Alternative identifiers and contributing source records remain available. Your product gets a common reference without discarding provider IDs or rewriting authoritative source data. An issuer’s LEI identifies a legal entity, not the instrument it issued.

Same instrument does not mean same listing.

An instrument can have several trading lines. Keep its identity separate from venue, currency, and other source-defined dimensions. Resolve the instrument, then filter the selected Catalog’s records for the trading context your product needs. A lookup can return several records; even ISIN, MIC, and currency are not universally sufficient to identify one listing.

When you need one exact source record, address it by {catalog, record_key}. Its key stays source-owned—not a universal instrument ID or a vendor-independent listing ID.

Identity connects. Coverage tells you what is available. Knowing that two identifiers refer to the same instrument does not mean every Dataset contains it. Combine identity resolution with remixed Dataset coverage to join provider information, discover specialist data, and offer the right product features for that entity.

Changing claims, priorities, and exact references

Resolution uses source-supplied claims and configured policies, not name matching or an assumed global identifier table. Source updates and withdrawals recompute affected mappings; downstream remixers follow the published Identity Catalog. Conflicts and rebalances remain inspectable.

A leading identifier can change as claims or priorities change; it is not a promised permanent ID. Exact source-record references also require handling missing or reused keys. Mapping establishes neither data completeness nor access rights.

03 / REMIX YOUR UNIVERSE

Your main Dataset. A map to everything else.

Remixing is how you build the Dataset your product starts from: the entities you cover, the information you bring together, and the specialist data your users can explore next.

Define your universe. Enrich it your way.

Choose which input Datasets determine membership in your main Dataset. Resolve their records to shared entities, combine fields with your own source priorities and rules, and enrich them from other inputs. Remixing materializes the resulting Catalog and updates affected entities as inputs change, retaining provenance and access requirements.

Discover coverage. Offer the right next step.

Publish a per-entity overview of membership in configured input Catalogs, including their record keys and lifecycle state. Track a specialist Dataset’s coverage without copying all its fields into your main model. Your product can discover which additional Datasets cover an entity and link to their information.

A useful link—not an empty section.

Two instruments may both be classified as STRUC_PRODUCT. Only one is covered by your structured-products masterdata Dataset. Show the “Structured-product details” link for that instrument—not for every instrument with the same category. Base the link on actual Dataset membership and the user’s access, with the lifecycle state appropriate to your product.

04 / THE SERVICES

One Gateway. Purpose-built services.

Your team builds against one interface.

The Gateway brings the services together behind one customer-facing interface. It authenticates requests, applies access rules, and routes them to the right service. Your developers use a consistent, schema-generated SDK for structured records, current values, history, and calculated figures—not a separate integration for every service.

Actual Gateway Explorer overview listing datasets and their configured Catalog, Latest, Timeseries, and Keyfigures services.
Your datasets and services, in one place. The built-in Explorer lists datasets and their configured services. Actual screenshot from our local evaluation deployment—not a service health dashboard or a catalogue of bundled data rights. Click to view full size.

READER / CATALOG ACCESS

Find and retrieve your structured data.

Turn a Catalog into a usable application interface. Retrieve selected fields, look up records by keys or identifying dimensions, and expose configured search and discovery capabilities.

  • Record and field retrieval; identity-aware lookup through the platform.
  • Autocomplete, text search, facets, and numeric filters.
  • Configured search projections and access rules.

Build an instrument finder, an issuer profile, or a searchable fundamentals dataset.

LAST / LATEST VALUES

The current picture, continuously updated.

Read the latest available values, then follow updates through a subscription. Shared upstream processing serves multiple consumers without duplicating the entire ingestion path.

  • Current snapshots followed by typed updates—not just the last traded price.
  • Applicable real-time, delayed, and end-of-day delivery policies.
  • Continuity and gap information alongside the data.

Build a watchlist, a quote page, or a customer-facing live API.

TIMESERIES / HISTORY & AGGREGATION

Tick history and OHLC. Built to scale.

Query individual historical observations and open/high/low/close candles on the same platform. Partitioned storage, compressed pages, and indexed range reads keep queries focused on the data they need.

  • Tick-level history, OHLC aggregation, historical queries, and live-following interfaces.
  • Explicit coverage and gaps; supported intervals follow the configured model.
  • Separate query and materialization workers: add serving capacity without repeating historical processing.

Build charts, historical analysis, or a vendor’s historical-data API.

KEYFIGURES / CALCULATIONS & SCREENING

Turn data into useful calculations.

Combine Catalog information with selected price inputs. Define calculation logic and expose results for individual instruments or searchable result sets.

  • Configurable calculations, input selection, and derived numeric values.
  • Filtering, sorting, and screening with price-freshness, availability, and access policies.
  • Midpoint and spread are implemented examples; additional financial formulas and their inputs are scoped separately.

Build a product screener or a customer-specific calculated dataset.

OBSERVATION / WATCHES & TRIGGERS

Turn changing data into useful notifications.

Build persistent watches around the conditions your users care about.

  • Durable task state, trigger transitions, and protection against rapidly oscillating conditions.
  • Task state and pending notifications committed together; delivery retries use a durable outbox.
  • Historical recovery produces separate summaries, not misleading live alerts, and does not overwrite the current live state.

Your application supplies instrument resolution, typed rules, and notification delivery. The reusable core is implemented; a ready-made Gateway observation API or email/push integration is not implied.

Build price alerts, threshold watches, or customer-specific notifications that bring users back at the right moment.

Actual Gateway Explorer: a Microsoft name search with stock and exchange-traded-product facet counts.
Search and discover in the built-in Explorer. Name search and asset-class facets in the Keyfigures view of our local evaluation deployment. Actual screenshot, cropped to the search controls; no quote values are shown. Click to view full size.

Corporate actions. Consolidated and applied.

Consolidate dated corporate actions and distributions across sources with deterministic source priorities and retained provenance. Request raw values or opt into split adjustment through the Gateway for supported snapshot, live, and time-series fields. Confirmed split ratios adjust prices and quantities at delivery; stored source data stays unchanged.

Raw is the default. Distributions are modeled and consolidated, but are not yet applied as a total-return adjustment. Feed coverage and the financial rules required by your product remain part of the integration scope.

Delivery semantics and integration boundaries

Catalog import cursors advance after writes complete; replayed writes are idempotent. Immutable payloads precede manifest publication. These are scoped semantics, not a blanket exactly-once promise. Ordering is source-local; SDK reconnect delivery is at least once. Clients handle gaps and replacement values according to the selected API contract.

Catalog search exposes a configured projection to holders of its search grant; per-record detail permissions do not filter that projection. Review its fields before enabling it. Keyfigures search checks the caller’s permitted inputs before returning results. Search freshness and ranking follow the selected service’s contract; there is no universal guarantee of exact global live ranking.

05 / CONNECT

Your feeds. On a shared foundation.

You should choose feeds for your product—not to fit our connector list. We implement or adapt the parsers, protocol handling, and mappings for the providers you need, using the shared ingestion, recovery, storage, and delivery infrastructure already in place. Your pilot defines the exact interfaces and the integration work still required.

Explore existing adapters and test sources

Starting points with local automated test coverage—not the limit of what we can integrate, or a claim of production certification.

Implemented interfaces — local test coverage, not production certification
Source Interface & scope Important boundary
Synthetic quote source (SIM) Configurable instrument count and messages per second, independently per simulated venue. Repeatable integration and load tests without vendor market data. The configured rate is requested load, not a measured throughput guarantee.
IEX TOPS protocol decoding and replay; source integration with a compatible endpoint. Development replay and protocol tests. No claim of certified live exchange connectivity.
Xetra All-tradable-instruments semicolon-delimited full file over HTTP. Reference data, not an order-book feed. Column and venue validation.
ESMA FIRDS Full and delta publications; ZIP/XML ingestion. Publication completeness and cursor handling; not a corporate-action engine.
GLEIF LEI2 Golden Copy full and delta ingestion. Legal-entity reference data, not market prices.
OpenFIGI Identity-driven HTTP mapping API worker. Identifier enrichment, not a price feed or a universal identity guarantee.

06 / ENGINEERING TARGET

Two million instruments.
One fast CPU core.

Our goal: handle a feed of 2 million instruments at 200,000 updates per second, including ingestion, latest-state maintenance, and time-series processing, on one fast CPU core.

Only the separate S3 storage server’s resources are excluded. OpenMDTA’s encoding, compression, and client-side storage work remain in scope.

Engineering target—not a measured end-to-end result or an SLA. Hardware, payloads, persistence settings, and recovery load must be specified and measured before claiming this target is met.

07 / BUILD

Typed data. Familiar code.

Schema-generated SDKs carry your fields into your developers’ workflow. This example from our generated Globex SDK reads a quote sample with typed bid/ask, trade, and custom fields. Exchange messages stay grouped; prices remain exact decimal strings.

import { connect, selector, type ConnectOptions } from "@globex/market-data";

export async function readQuoteSample(options: ConnectOptions, symbol: string) {
  const connection = await connect(options);
  try {
    const requested = selector.us(symbol).at("IEX");
    const result = [];
    for await (const msg of connection.stream({
      selector: requested,
      blocks: ["BidAsk", "Trade", "ClientTradeAttributes"],
    })) {
      if (msg.selector !== requested) throw new Error("response belongs to another selector");
      console.log("yes, it is what we requested");
      result.push({
        phase: msg.phase,
        requestedSelector: msg.selector.expression,
        sourceRecord: msg.sourceRecord,
        messageId: msg.messageId.toString(),
        bid: msg.fields.bidAsk?.bid?.price.toString() ?? null,
        ask: msg.fields.bidAsk?.ask?.price.toString() ?? null,
        trade: msg.fields.trade?.price.toString() ?? null,
        clientTradeAttributes: msg.fields.clientTradeAttributes ?? null,
        presentFields: [...msg.fieldIterator()].map((field) => field.name),
      });
      if (result.length === 2) break;
    }
    return result;
  } finally {
    await connection.close();
  }
}

08 / OPERATE

Durable storage. Replaceable services.

OpenMDTA’s durable state lives in S3-compatible storage, not in an irreplaceable service instance. Restart or replace services, reconnect them to the same storage, and resume from persisted state. Reliable storage is the foundation of recovery—without an OpenMDTA-specific backup-and-restore procedure for each service.

Keep deployment configuration and credentials available independently. Storage must provide the required consistency and conditional-write semantics; availability, retention, and data protection remain part of the deployment design.

Access & isolation

Data credentials implement scoped grants, audience/origin checks, and limited lifetime. Deployment isolation, key management, and network controls need an agreed configuration and review. No independent penetration-test report, security certification, or audited tenant-isolation guarantee is published here.

Continuity & visibility

Explicit gaps and replay mechanisms are implementation features. Multi-region HA, storage disaster recovery, restart times, stale-data monitoring, reconciliation, and bad-tick handling must be demonstrated for the deployment. No public SLA or 24/7 support coverage is promised.

Rights & provenance

Connectors do not grant exchange or vendor data rights. Agree permitted use, redistribution, and entitlements before connecting licensed data. Code provenance, dependency licenses, and commercialization evidence belong in the technical and contractual review; this page is not a legal clearance statement.

Control & handover

Choose the operating model that fits your team:

  • We run it on our infrastructure. We provide the environment and operate OpenMDTA.
  • We run it on your infrastructure. We operate OpenMDTA in your environment with agreed access and responsibilities.
  • You run it on your infrastructure. Your team handles deployment and operations, with optional integration and support from us.

Define data export, configuration portability, source access, operating documentation, and handover tests before commitment. Source escrow is not promised.

Our goal: independence you can demonstrate.

Give an external engineering team only the public repositories and documentation. Can they build, deploy, ingest a feed, generate an SDK, redeploy services against existing storage, upgrade, and operate the system without contacting OpenMDTA? That is our acceptance test for independent self-hosting.

A goal, not a completed validation. This does not announce a public repository release or claim that a team has passed the test. Required infrastructure and feed permissions remain the operator’s responsibility.