A profitable backtest can lose its edge in live trading when spreads, slippage, latency, partial fills, rejected orders, and fees are ignored. This guide shows how to audit a trading bot’s execution quality, estimate hidden transaction costs, and turn trade-log evidence into practical changes to order logic, position sizing, and broker selection.
Overview
Trading bot slippage is the difference between the price a strategy expects and the price at which an order actually executes. It is only one part of execution cost. A complete review should also account for the bid-ask spread, market impact, commissions or other broker charges, delayed order submission, partial fills, rejected orders, and trades that were never completed.
The most useful comparison is not simply “backtest return versus live return.” Instead, compare the bot’s intended orders with its realized executions. This helps separate strategy performance from implementation quality. A signal may be sound while the order process is too slow or too aggressive. Conversely, a bot may appear to have a timing problem when the main issue is a wide spread or an order type that is poorly suited to the instrument.
Keep a record of the strategy version, instrument, side, order type, intended quantity, signal timestamp, submission timestamp, broker acknowledgment, fill timestamps, fill prices, and any rejection or cancellation reason. The same record can support a monthly review, a broker comparison, and a release audit after changes to the bot.
| Metric | What it measures | Useful calculation or review |
|---|---|---|
| Implementation shortfall | The difference between the decision price and the realized execution outcome. | For a buy, compare the decision or arrival price with the volume-weighted fill price; reverse the sign logic for a sell. |
| Spread cost | The cost of crossing the bid-ask spread. | Compare the fill price with the midpoint at the time the order was submitted or received. |
| Market impact | The price movement associated with the bot’s order interacting with available liquidity. | Compare the fill sequence with the midpoint or quoted prices before and during execution. |
| Fill rate | How much of the requested quantity was executed. | Filled quantity divided by submitted quantity, tracked by order type and instrument. |
| Rejected orders | Orders that failed before reaching the market or execution venue. | Count rejections and group them by error, time, symbol, account state, and order type. |
| Delayed execution | Time lost between signal, submission, acknowledgment, and fill. | Measure each timestamp interval separately rather than using one combined latency number. |
How to estimate execution quality
Use a repeatable audit rather than reviewing only the largest losing trades. A practical workflow is:
- Freeze the review period. Select a defined sample, such as one strategy release, one month, or a fixed number of completed trades. Do not mix paper trades, simulated fills, and live executions without labeling them.
- Reconstruct the order timeline. Capture the signal time, decision price, order creation time, API submission time, broker acknowledgment, exchange or venue acknowledgment when available, each fill, cancellation, and final status. Your broker API may expose only some of these events.
- Normalize the direction. A higher price is normally unfavorable for a buy and favorable for a sell. Convert costs into a common unit, such as currency, cents per share, basis points, or ticks, so different instruments can be compared.
- Calculate the realized fill price. For multiple fills, use a volume-weighted average price: VWAP = sum of fill price × fill quantity ÷ total filled quantity.
- Measure arrival-price shortfall. For a buy, a simple estimate is (realized VWAP − arrival price) × filled quantity. For a sell, use (arrival price − realized VWAP) × filled quantity. Add explicit fees separately unless your reporting system already includes them.
- Split the result into components. Compare the midpoint and quoted spread at the relevant timestamps. Then inspect whether the market moved before submission, during latency, or while the order was being filled. This prevents all unfavorable movement from being labeled as slippage.
- Review incomplete and rejected orders. A partial fill can create exposure different from the strategy’s intended position. A rejection can produce an opportunity cost, a late replacement order, or an unintended risk limit breach. Track these outcomes instead of excluding them as exceptions.
- Segment the findings. Group results by symbol, average volume, time of day, volatility regime, order type, order size, buy versus sell, and strategy signal. Aggregates often hide a problem that affects only one market condition.
For a practical trade-log template, include: trade ID, bot version, symbol, side, intended quantity, filled quantity, order type, limit or stop price, decision price, quote midpoint, bid, ask, signal timestamp, submission timestamp, acknowledgment timestamp, first fill timestamp, final fill timestamp, average fill price, fees, rejection code, cancellation reason, and realized shortfall. Keep the original raw events as well as the cleaned summary.
When a bot is connected through a broker API, also document the broker’s timestamp conventions. Local computer time, server time, and market timestamps may not be directly comparable. Consistent time synchronization and an explicit timezone field make latency analysis more reliable.
Inputs and assumptions
An execution audit is only as useful as its inputs. Before calculating, define the following assumptions:
- Reference price: Decide whether the benchmark is the signal price, arrival price, quote midpoint, last trade, or a modeled fair value. Use the same benchmark across comparable trades.
- Cost scope: State whether the estimate includes commissions, exchange or regulatory charges, borrow costs, funding, and data or platform fees. Keep direct execution costs separate from broader operating expenses when possible.
- Time granularity: Record timestamps precisely enough to distinguish signal delay, API delay, broker processing, and time spent waiting for a fill.
- Market data quality: Note whether bid, ask, and midpoint data are real-time, delayed, consolidated, or reconstructed. A stale quote can make an otherwise accurate fill appear unfavorable.
- Partial-fill treatment: Calculate costs on the filled quantity, but separately record the unfilled quantity and its consequence. Do not treat an unfilled order as a zero-cost trade.
- Counterfactual benchmark: For canceled or rejected orders, define what you will compare: the next available quote, a replacement execution, the strategy’s next signal, or no-trade outcome. Label this as an estimate rather than a realized cost.
To estimate the effect on a strategy, use net result = gross trading result − execution costs − explicit fees. Execution costs should be measured in the same currency and period as the strategy’s returns. A small cost per trade can become material when turnover rises, so review both cost per share or contract and total cost as a percentage of traded value.
Backtests should model conservative assumptions rather than a single ideal fill. A useful validation process compares several scenarios, such as midpoint execution, spread-crossing execution, and a defined additional slippage allowance. More detail on separating simulated performance from live behavior is available in this backtesting framework.
Worked examples
Example 1: Estimating spread cost and shortfall
Assume a bot wants to buy 500 shares. At the decision time, the quoted bid is $49.98 and the ask is $50.02, so the midpoint is $50.00. The order is submitted as a marketable order and fills in two pieces at $50.03 and $50.05. Assume the average fill price is $50.04 and explicit fees are excluded from this example.
The approximate spread-related comparison to the midpoint is ($50.04 − $50.00) × 500 = $20. The arrival-price shortfall is also $20 if the arrival price is defined as the $50.00 midpoint. However, the full spread was $0.04, or $20 for 500 shares, so the extra $0.04 of adverse movement beyond the midpoint may indicate market movement, queue position, or market impact rather than a simple half-spread cost. The audit should therefore inspect the quote and fill timestamps instead of assigning the entire result to slippage.
Example 2: Partial fills and delayed execution
Suppose a strategy signals a 1,000-share purchase at an arrival price of $20.00. The bot submits 1,000 shares, receives an acknowledgment after a delay, fills 600 shares at an average of $20.02, and cancels the remaining 400 shares. The realized shortfall on the filled quantity is ($20.02 − $20.00) × 600 = $12.
The $12 is not the complete execution story. The log should also show why 400 shares remained unfilled, whether the strategy ended with an undersized position, and what price was available when the order was canceled. If the bot later submits a replacement order, measure that order separately and connect both records with the same parent trade ID.
Example 3: Comparing two order approaches
Imagine the same strategy uses a marketable order in one test group and a limit order in another. The marketable order may show a higher fill rate but larger spread and impact costs. The limit order may reduce price concession but increase waiting time, partial fills, and missed signals. The better approach is not automatically the one with the lowest cost per filled share; compare net strategy results, completed position size, opportunity cost, and risk created by delayed or incomplete exposure.
For a broader review of bot-level results, pair execution metrics with drawdown, win rate, and other performance measures in the trading bot performance metrics reference. Execution quality is an implementation input, not a replacement for strategy evaluation.
When to recalculate
Re-run the audit after every material strategy or infrastructure change, including a new broker, API, exchange connection, instrument, order type, position-sizing rule, data feed, or hosting location. It is also worth reviewing after a sharp increase in order size or turnover, because orders that were small relative to available liquidity may behave differently at larger scale.
For routine oversight, review a defined sample monthly and perform a deeper comparison after each strategy release. Recheck broker fee schedules, order-type terminology, market-session rules, and available API fields at least every six months, or sooner after a broker, exchange, or API change. These details affect both calculations and the meaning of trade-log fields.
Use the results to make one controlled change at a time. Possible actions include reducing order size, adding a spread or liquidity filter, separating opening and closing-session logic, imposing a maximum acceptable spread, improving timestamp collection, using a limit-price boundary, or pausing when market data and order acknowledgments diverge. Test each change in paper trading before live deployment when practical, then compare live results with the same benchmark used in the original audit.
Finally, keep an execution-quality dashboard with median and high-percentile shortfall, fill rate, rejection rate, average signal-to-submission delay, average submission-to-fill delay, and total transaction costs. Revisit it whenever your broker, strategy, volume, or market conditions change. A bot does not need perfect fills, but you should know which costs are expected, which are controllable, and when they are large enough to change the strategy’s decision.