Pitfield St

6Methodology

Every number on this site comes from a formula on this page. Where we had to make a choice, such as a convention, a filter or an assumption, it is written down here too.

6.1Forward and rate extraction

We do not plug in a Treasury yield and a dividend estimate. Both are backed out of the chain, per expiry, from put–call parity:

C(K)−P(K)=e−rTF−e−rTKC(K) - P(K) = e^{-rT}F - e^{-rT}K

This is linear in K with slope β = −e−rT and intercept α = e−rTF, so a single regression of the call-minus-put mid on strike gives both quantities:

r=−ln⁡(−β)T,F=−αβr = -\frac{\ln(-\beta)}{T}, \qquad F = -\frac{\alpha}{\beta}

The forward recovered this way already includes dividends and borrowing costs, which is why we extract it instead of assuming them. A wrong dividend assumption is a common reason fitted surfaces come out lopsided.

We use strikes in roughly the 0.85–1.15 moneyness band, where both legs are liquid, and fit twice: once against a spot-referenced band to obtain a first forward, then again against a band referenced to that forward. The regression R² is recorded per expiry and the expiry is rejected below 0.99. A poor fit almost always means bad quotes.

A precision note on the rate

The forward is recovered essentially exactly at every maturity. The rate is not. Because r = −ln(−β)/T, any slope error is amplified by 1/T: at one week that factor exceeds 50, so tick rounding alone moves the extracted rate by tens of basis points, while at one year the same rounding moves it by less than one. The forward is the number this step exists to produce and it is reliable throughout; the short-dated rate is a by-product and is published as such. Downstream, the rate enters only as a discount factor, where the sensitivity is second order.

6.2Surface calibration

We work in total implied variance w = σ²T against log-moneyness k = ln(K/F), using Gatheral's raw SVI parameterisation:

w(k)=a+b[ρ(k−m)+(k−m)2+σ2]w(k) = a + b\left[\rho(k-m) + \sqrt{(k-m)^2 + \sigma^2}\right]

subject to

b≥0,∣ρ∣<1,σ>0,a+bσ1−ρ2≥0b \ge 0, \quad |\rho| < 1, \quad \sigma > 0, \quad a + b\sigma\sqrt{1-\rho^2} \ge 0

The final constraint is exactly the statement that total variance is non-negative everywhere. Setting w′(k) = 0 puts the minimum where the wing slopes b(ρ ± 1) balance:

k−m=−ρσ1−ρ2  ⟹  wmin⁡=a+bσ1−ρ2k - m = -\frac{\rho\sigma}{\sqrt{1-\rho^2}} \;\Longrightarrow\; w_{\min} = a + b\sigma\sqrt{1-\rho^2}

Mid prices are inverted to implied volatilities with Black-76 on the extracted forward, by Brent search on a bracketed interval; a contract whose price falls outside the no-arbitrage bounds has no implied volatility and is dropped and counted rather than coerced to a number. Each expiry is then calibrated by constrained weighted least squares (SLSQP), seeded from the previous day's parameters, surfaces move slowly, and a warm start keeps the parameter time series stable instead of hopping between equivalent optima.

Residuals are weighted by vega, never equally. Equal weighting lets illiquid wings dominate a fit that matters most at the money, and the wing failure mode that actually occurs is bias rather than noise: wing markets are wide, and the mid of a wide market is systematically off.

6.3Arbitrage conditions

Both conditions are checked on every surface, every day, and the results, including the failures, are published.

Butterfly, after Durrleman

g(k)=(1−k w′(k)2w(k)) ⁣2−w′(k)24(1w(k)+14)+w′′(k)2g(k) = \left(1 - \frac{k\,w'(k)}{2w(k)}\right)^{\!2} - \frac{w'(k)^2}{4}\left(\frac{1}{w(k)} + \frac14\right) + \frac{w''(k)}{2}

g(k) ≥ 0 everywhere is equivalent to the fitted smile implying a non-negative probability density. We evaluate it on a dense grid and publish min g(k) for every expiry, so a reader sees the margin rather than a boolean.

Calendar

∂w(k,T)∂T≥0for all k\frac{\partial w(k,T)}{\partial T} \ge 0 \quad \text{for all } k

Total variance must not decrease with maturity at fixed log-moneyness.

Where the conditions are evaluated

Both are checked only across the log-moneyness range where that expiry had quotable contracts, and the surface is published only over the same range. Checking further out would flag artefacts rather than real violations. A seven-day expiry has no quotes thirty percent out of the money, so its fitted value there is pure extrapolation, and comparing that extrapolation against a longer expiry that does have quotes reports a crossing between one real curve and one invented one. We claim arbitrage-freedom only where we publish a number, and the rendered surface carries a ragged edge for the same reason: that edge is the data support.

6.4Greeks and units

Everything is priced on the extracted forward with Black-76, which removes the dividend problem entirely because dividends are already inside F.

d1=ln⁡(F/K)+12σ2TσT,d2=d1−σTd_1 = \frac{\ln(F/K) + \tfrac12\sigma^2T}{\sigma\sqrt{T}}, \qquad d_2 = d_1 - \sigma\sqrt{T} Δ=e−rTN(d1),Γ=e−rTφ(d1)FσT,V=e−rTFφ(d1)T\Delta = e^{-rT}N(d_1), \quad \Gamma = \frac{e^{-rT}\varphi(d_1)}{F\sigma\sqrt{T}}, \quad \mathcal{V} = e^{-rT}F\varphi(d_1)\sqrt{T} vanna=−e−rTφ(d1)d2σ,volga=V d1d2σ\text{vanna} = -\frac{e^{-rT}\varphi(d_1)d_2}{\sigma}, \qquad \text{volga} = \mathcal{V}\,\frac{d_1 d_2}{\sigma}

Vanna is ∂𝒱/∂F and volga is ∂𝒱/∂σ. Differentiating vega with respect to the forward and using d₂ = d₁ − σ√T:

∂V∂F=e−rTT φ(d1)[1−d1σT]=−e−rTφ(d1)d2σ\frac{\partial \mathcal{V}}{\partial F} = e^{-rT}\sqrt{T}\,\varphi(d_1)\left[1 - \frac{d_1}{\sigma\sqrt{T}}\right] = -\frac{e^{-rT}\varphi(d_1)d_2}{\sigma}

Theta and charm are one-day finite differences with a bump of 1/365 years, documented here because the bump size is part of the number. Closed forms exist; a bump is easier to verify against a repriced option, and the test suite checks it against the analytic result.

Published units. Half of all greek disagreements are unit disagreements.
Vegaper volatility point, i.e. per 0.01 of σ
Theta, charmper calendar day
Delta, gamma, vanna, volgaper unit of forward, discounted to today

6.5Derived metrics

Constant-maturity implied volatility

Interpolation is in total variance, then converted back:

w(T∗)=w(Ti)+T∗−TiTi+1−Ti[w(Ti+1)−w(Ti)],σ∗=w(T∗)/T∗w(T^\ast) = w(T_i) + \frac{T^\ast - T_i}{T_{i+1} - T_i}\left[w(T_{i+1}) - w(T_i)\right], \qquad \sigma^\ast = \sqrt{w(T^\ast)/T^\ast}

Total variance is the quantity that is additive in time. Interpolating volatility directly is wrong and produces a term structure that sags between listed expiries. We do not extrapolate beyond the last listed expiry; a target outside the quoted range returns null.

Skew

The 25-delta risk reversal and butterfly are computed at the strikes where delta genuinely equals 25, solved on the fitted smile: the strike at a given delta depends on the volatility there, which depends on the strike, so it is a root-find rather than an ATM-volatility approximation. The at-the-money skew slope is taken analytically from the fit:

∂σ∂k∣k=0=w′(0)2Tσ(0)\frac{\partial \sigma}{\partial k}\bigg|_{k=0} = \frac{w'(0)}{2T\sigma(0)}

Variance risk premium

VRP(t)=IV30(t)−RV30(t→t+30)\mathrm{VRP}(t) = \mathrm{IV}_{30}(t) - \mathrm{RV}_{30}(t \to t+30)

Only computable thirty days after the fact. It is published lagged and labelled as such. Showing it any earlier would mean using information that didn't exist yet.

6.6Realized volatility

Five estimators at 5, 10, 21, 63 and 252 trading days, annualised on 252 days: close-to-close, Parkinson, Garman–Klass, Rogers–Satchell and Yang–Zhang.

σYZ2=σovernight2+k σoc2+(1−k) σRS2,k=0.341.34+N+1N−1\sigma^2_{YZ} = \sigma^2_{\text{overnight}} + k\,\sigma^2_{\text{oc}} + (1-k)\,\sigma^2_{RS}, \qquad k = \frac{0.34}{1.34 + \frac{N+1}{N-1}}

They are published side by side because they measure different things. Parkinson, Garman–Klass and Rogers–Satchell use only intraday prices and are structurally blind to the overnight gap, so on gappy data they sit below close-to-close; Yang–Zhang is the one that adds the gap back. Rogers–Satchell is additionally drift-independent, which the other two are not.

We also publish σP / σCC, the ratio of the range-based estimator to close-to-close. It shows how much of the day's movement happens inside the session, and few sources publish it.

6.7Chain aggregates

GEX(K)=Γ(K)⋅OI(K)⋅multiplier⋅S2⋅0.01\mathrm{GEX}(K) = \Gamma(K)\cdot \mathrm{OI}(K)\cdot \text{multiplier}\cdot S^2 \cdot 0.01

Dollar gamma per 1% move in spot.

The sign convention is an assumption, not a fact

The common convention treats dealers as long gamma from calls and short from puts. That assumes customers buy puts and sell calls, a generalisation about flow, not something anyone can observe, since no dealer's book is public. It is implemented as a configuration flag, the chosen convention travels with the number in the output schema, and it is restated on every page that displays a gamma figure.

Open interest compounds the problem: it is previous-day settled, so today's reported figure reflects yesterday's positions. Every metric built on it is timestamped to the open interest's effective date, not to the snapshot date.

6.8Filtering and exclusions

Before calibration a contract is dropped if it is:

Nothing is dropped silently. Counts are published per expiry and in aggregate on the quality page, because the share of a chain excluded from a fit is part of the result: a surface fitted to a third of a chain tells you less than one fitted to all of it.

6.9Equities

The equity archive covers every active, tradable US stock and fund listed on an exchange, about thirteen thousand symbols, leaving out over-the-counter names, warrants, rights, units and preferreds. Bars come from the SIP feed, the consolidated tape of every exchange and off-exchange venue, served fifteen minutes delayed and captured after extended hours end at 20:00 New York time.

Raw daily and five-minute bars are archived by session, append-only, in the same discipline as the option chains. Returns are computed from a separate two-year panel adjusted for splits and dividends. That panel cannot be append-only, since every dividend restates the history before it, so each night the last fortnight is fetched again for every symbol and any symbol whose past closes no longer match is refetched in full.

The published history reaches back to January 2016. Before 19 October 2026, when our own capture began, it comes from a separate library built from the same two sources, the consolidated tape and SEC filings, and put through the same rules. That library is described below.

Company size and industry

Shares outstanding come from SEC filings: the cover-page count from each company's latest report, or weighted diluted shares where only per-class counts are filed. Market value is that count times the close, as traded. In the history, the count is the latest one public on the day. Every day, the month's dollar volume is checked against the market value it implies, and a stock outside a wide band is left out, because there the count and the traded line can't be the same shares. Buckets are mega above $200bn, large $10bn to $200bn, mid $2bn to $10bn, small $300m to $2bn and micro below. A company with more than one listed share class is counted once, by its most traded class. Industries are the SEC's classification codes mapped by hand to eleven sectors.

Floors, and no lookahead

Aggregates of daily returns include a stock only if it closed at or above $1 the day before and traded at least $250,000 a day over the previous month. Factor spreads use $5 and $1m. Every characteristic used to group stocks for day t, size, momentum, volatility and liquidity, is measured at the close of day t−1.

The overnight and session split

1+rt=OtCt−1⏟overnight⋅CtOt⏟session1 + r_t = \underbrace{\frac{O_t}{C_{t-1}}}_{\text{overnight}} \cdot \underbrace{\frac{C_t}{O_t}}_{\text{session}}

Each day's return factors exactly into the move while the market was shut and the move while it traded. Both are averaged across a bucket each day and compounded separately.

Factor spreads

st=1∣Q5∣∑i∈Q5(t−1)ri,t  −  1∣Q1∣∑i∈Q1(t−1)ri,ts_t = \frac{1}{|Q_5|}\sum_{i \in Q_5(t-1)} r_{i,t} \; - \; \frac{1}{|Q_1|}\sum_{i \in Q_1(t-1)} r_{i,t}

The equal-weighted return of the top fifth of stocks on a characteristic, sorted at the previous close, less the bottom fifth. Size is small minus large, momentum is the 12-minus-1 month return, reversal is the previous month's return with losers long, and low volatility is 63-day volatility with the calmest long. Before costs, and published as descriptions of the market rather than as strategies.

The history before our capture

From January 2016 to 16 October 2026 the history keeps every company that was listed at the time, including the thousands since delisted, taken over or gone bust, as one series per company across ticker changes. Leave those out and small caps look far better than they were, since that is where companies fail. Each company is sized by the share count it had filed by then, not today's. When a company leaves its exchange, the next session carries its holders' last return: the cash price in a cash takeover, and a 35 per cent loss for a delisting on performance or one nobody explained, the convention after Shumway (1997). That figure is an assumption, and it is the one number in the history nobody observed. From 19 October 2026 the history is our own capture. Each session's figures are recorded the night they are first computed and never recomputed, so a company that delists later stays in the sessions it traded in.

6.10Intraday

Options

The whole chain is captured at a quarter past and a quarter to each hour from 09:45 to 15:45 New York time, never before ten minutes after the open, and each snapshot is fitted by the same code as the close, warm-started from the snapshot before. Time to expiry is counted in whole days, as in the daily record, which slightly overstates the remaining life of the shortest expiries late in a session and does not move the 30, 60 and 90 day figures. The last point of each day is the after-the-close record itself.

Equities

Five-minute bars from 04:00 to 20:00. The intraday profile uses the regular session, 09:30 to 16:00, and only stocks with a trade in at least 60 of its 78 slots. The 16:00 slot holds the closing auction, together with the first five minutes of after-hours trading. Session volatility is measured from five-minute returns within the session, so the overnight gap is excluded:

σsession=252∑j=278ln⁡2 ⁣(CjCj−1)\sigma_{\text{session}} = \sqrt{252 \sum_{j=2}^{78} \ln^2\!\left(\frac{C_j}{C_{j-1}}\right)}

6.11Known limitations

6.12References

The papers the methods on this page come from.

  1. Black, F. (1976). The pricing of commodity contracts. Journal of Financial Economics, 3, 167–179.
  2. Gatheral, J. (2004). A parsimonious arbitrage-free implied volatility parameterization with application to the valuation of volatility derivatives. Global Derivatives & Risk Management, Madrid.
  3. Gatheral, J. and Jacquier, A. (2014). Arbitrage-free SVI volatility surfaces. Quantitative Finance, 14(1), 59–71.
  4. Durrleman, V. (2005). From Implied to Spot Volatilities. PhD thesis, Princeton University.
  5. Britten-Jones, M. and Neuberger, A. (2000). Option prices, implied price processes, and stochastic volatility. Journal of Finance, 55(2), 839–866.
  6. Carr, P. and Wu, L. (2009). Variance risk premiums. Review of Financial Studies, 22(3), 1311–1341.
  7. Parkinson, M. (1980). The extreme value method for estimating the variance of the rate of return. Journal of Business, 53(1), 61–65.
  8. Garman, M. B. and Klass, M. J. (1980). On the estimation of security price volatilities from historical data. Journal of Business, 53(1), 67–78.
  9. Rogers, L. C. G. and Satchell, S. E. (1991). Estimating variance from high, low and closing prices. Annals of Applied Probability, 1(4), 504–512.
  10. Yang, D. and Zhang, Q. (2000). Drift-independent volatility estimation based on high, low, open, and close prices. Journal of Business, 73(3), 477–492.
  11. Jegadeesh, N. and Titman, S. (1993). Returns to buying winners and selling losers. Journal of Finance, 48(1), 65–91.
  12. Lou, D., Polk, C. and Skouras, S. (2019). A tug of war: Overnight versus intraday expected returns. Journal of Financial Economics, 134(1), 192–213.
  13. Shumway, T. (1997). The delisting bias in CRSP data. Journal of Finance, 52(1), 327–340.
  14. Blume, M. E. and Stambaugh, R. F. (1983). Biases in computed returns: An application to the size effect. Journal of Financial Economics, 12(3), 387–404.
  15. Newey, W. K. and West, K. D. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708.
  16. Politis, D. N. and Romano, J. P. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428), 1303–1313.
  17. Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate. Journal of the Royal Statistical Society, Series B, 57(1), 289–300.