System requirements

Functional:

1. real-time market data ingestion
2. order management
3. trading signal generator
4. portfolio optimizer
5. prime brokerage connection
6. real-time monitoring for market data, order executions, and brokerage connections
7. backtesting or replaying a past trading session

Non-Functional:

1. Low latency in market data ingestion and order executions
2. Risk control: on detection of anomalies, pause the system or take preventive actions

Capacity estimation

Depending on the types of security instruments being traded, the amount of data may vary.

1. for stock equities, raw market data of one trading day typically is of tens of GB or hundreds of GB.
2. for options, one day of raw data may be at level of TB.

Real-time market data may arrive at any time during trading hours. Two piece of data may be nanoseconds or sub-nanoseconds apart on average.

API design

The system is fully automated, but occasionally we need manual intervention. Also, we need authentication APIs.

Authentication:

int auth(user_profile)

returns a status code indicating auth result

Manual intervention:

int change_system_status(new_status)

returns the new system status

this should be a state machine design

long place_order(order)

returns order id of the newly placed order, or -1 if failed

int cancel_order(order_id)

returns order status after cancel request is processed

we also need a killswitch API to cancel everything and put the system to stopped mode, in case anything bad happened

Database design

We write executions to database. These are highly structured data so we use SQL.

orders: an order may have zero to many fills

uid, security_id, price, qty, side, type, status, filled_qty, broker_id, created_at, last_updated_at

orders_history:

similar to orders, but tracks the history of every order data change

fills: to store every fill

uid, security_id, order_id, fill_price, filled_qty, broker_id, created_at

securities:

uid, type, open, close, high, low, exchange_id, country_id

portfolio:

date, security_id, qty, broker_id

High-level design

See diagram

Request flows

1. signal generator and order manager subscribes to market data
2. market data component ingests market data, and publishes data to subscribers
3. signal generator decides whether an order of a particular security shall be created
4. order manager takes the signal and creates an order or multiple orders based on current portfolio
5. order manager sends order to prime brokerage
6. order manager monitors existing orders status

Detailed component design

1. market data ingestion: this may be run as multi-thread per security ids if we don't need to compute real-time correlation between different securities.
2. market data monitoring may be run by a separate thread from the ingestion component.
3. execution monitoring may be run on a different thread from other execution services, such as order management and broker connection.
4. order management may run multiple thread, one for placing orders, one for updating status of existing orders, and one for sending order change requests to brokers.

Trade offs/Tech choices

many components run on multi-threading, we need to make sure resources are properly locked when they're shared between different threads. Such a multi-threading set up will slow down single execution, but ensures we can achieve more throughput.

the market data monitoring component is only interested in the market data ingestion, such that they may be combined as one subsystem to reduce complexity. Also, monitoring does not change any market data, such that we could avoid locking certain resources on the ingestion side to mitigate performance loss.

Failure scenarios/bottlenecks

The market data ingestion may be a latency bottleneck. To truly achieve subnanosecond-level latency which is required by high-frequency trading, we have to colocate part of the system to the servers at the market exchange, using hardware to capture data and send trading orders. Pure software system is not capable of achieving true high-frequency trading.

Future improvements

This system works for any types of security instruments. However, to trade multiple security instrument types, as they are all very different in nature, we should run multiple systems on separate machines, each being responsible for trading one type of instrument, such that reducing complexity and unnecessary falsy interference between different instruments.