Gasoline: Which variables are the strongest predictors for supply and demand?¶
This notebook ranks individual predictors for next-month U.S. supply and demand.
Supply and demand are separate targets.
FUEL = 'gasoline'
# Load the scientific stack used by the analysis. FUEL is the only setting that
# differs between the crude and gasoline versions of this notebook.
from pathlib import Path
import sys, hashlib, platform
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
import sklearn
# Show all ranking rows and keep the long, descriptive predictor names readable.
pd.set_option('display.max_rows', 60)
pd.set_option('display.max_colwidth', 100)
# Locate the shared project root whether the notebook is launched from this
# product folder or from a parent directory, then make both data loaders importable.
ROOT = next(p for p in [Path.cwd(), *Path.cwd().parents] if (p/'oil/us_snd_crude/crude_data.py').exists())
for fuel in ['crude', 'gasoline']:
sys.path.insert(0, str(ROOT/f'oil/us_snd_{fuel}'))
from crude_data import load_panel as load_crude, FLOWS as CRUDE_FLOWS
from gasoline_data import load_panel as load_gasoline, FLOWS as GAS_FLOWS
# Load monthly PADD-level histories for both products. The second product is
# needed later for cross-market predictors such as crude inputs for gasoline.
panels = {'crude': load_crude(), 'gasoline': load_gasoline()}
national = {}
for fuel, panel in panels.items():
# A national observation is valid only when all five PADD regions are present.
assert panel.groupby('month').padd.nunique().eq(5).all()
cols = (CRUDE_FLOWS + ['stock_kb', 'cdu_capacity_kbd']) if fuel == 'crude' else GAS_FLOWS + ['stock_kb']
# min_count=5 prevents a missing PADD from silently producing a partial U.S. total.
# asfreq also exposes any missing month in the national time series.
z = panel.groupby('month')[cols].sum(min_count=5).asfreq('MS')
assert z.notna().all().all()
# Crude flows already arrive as daily rates. Gasoline flows are stored as
# monthly thousand-barrel totals, so divide by the exact days in each month.
# Stocks are point-in-time levels and are converted from thousand to million barrels.
if fuel == 'gasoline':
for c in GAS_FLOWS:
z[c.replace('_kb', '_kbd')] = z[c] / z.index.days_in_month
z['stock_mb'] = z.stock_kb / 1000
z['change_mb'] = z.stock_mb.diff()
# These derived series summarize trade and the simplified physical balance.
# The core balance excludes adjustments, transfers and other secondary flows.
z['net_imports_kbd'] = z.imports_kbd - z.exports_kbd
z['core_balance_kbd'] = z.production_kbd + z.net_imports_kbd - z.demand_kbd
if fuel == 'crude':
# Refinery utilization compares actual crude inputs with operable capacity.
z['utilization'] = z.demand_kbd / z.cdu_capacity_kbd
national[fuel] = z
# Record the data range and source-file hash so an executed notebook identifies
# the exact local EIA snapshots used to produce its rankings.
manifest = []
for fuel in national:
path = ROOT/f'oil/us_snd_{fuel}/data/eia_observations.csv'
manifest.append({'fuel': fuel, 'start': national[fuel].index.min(), 'end': national[fuel].index.max(),
'months': len(national[fuel]), 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()})
How variable importance is measured¶
We test each variable separately while accounting for seasonal patterns (which variables are most important after controlling for season). Variables that reduce prediction errors the most are the most important.
The test uses six historical periods, with each prediction based only on earlier data.
We remove each variable from the model containing all inputs and refit to measure the information it adds alongside the other variables.
# Select the product being ranked and align the other product to the same calendar.
z = national[FUEL]
other_fuel = 'gasoline' if FUEL == 'crude' else 'crude'
other = national[other_fuel].reindex(z.index)
demand_name = 'Crude refinery inputs' if FUEL == 'crude' else 'Gasoline product supplied'
production_name = 'Crude production' if FUEL == 'crude' else 'Gasoline net production'
# Rank predictors separately for gross supply and demand. Gross supply includes
# domestic production and imports; exports remain an explanatory outflow.
targets = pd.DataFrame({
'Supply: production + imports': z.production_kbd + z.imports_kbd,
'Demand: ' + demand_name.lower(): z.demand_kbd,
})
# Twelve month indicators form the seasonality-only benchmark. Each later
# single-variable test asks whether a predictor improves on this calendar model.
X = pd.DataFrame(index=z.index)
for month in range(1, 13):
X[f'Calendar month {month:02}'] = (X.index.month == month).astype(float)
calendar = list(X.columns)
def history(name, series):
"""Add one-month and smoothed three-month lags without using forecast-month data."""
# Shift before rolling: for a forecast at month t, these features end at t-1.
X[f'{name} — previous month'] = series.shift(1)
X[f'{name} — previous 3-month average'] = series.shift(1).rolling(3).mean()
# Add recent histories of the product's main supply, trade and demand variables.
history(production_name, z.production_kbd)
history(FUEL.title() + ' imports', z.imports_kbd)
history(FUEL.title() + ' exports', z.exports_kbd)
history(demand_name, z.demand_kbd)
history('Supply (production + imports)', targets.iloc[:, 0])
history('Core balance (production + imports − exports − demand)', z.core_balance_kbd)
# Inventory features describe the latest level, recent changes, and the deviation
# from the same point one year earlier. Both levels in the last feature precede t.
X['Inventory level — previous month'] = z.stock_mb.shift(1)
X['Inventory change — previous month'] = z.change_mb.shift(1)
X['Inventory change — previous 3-month average'] = z.change_mb.shift(1).rolling(3).mean()
X['Inventory level minus year-earlier level'] = z.stock_mb.shift(1) - z.stock_mb.shift(13)
# Include the remaining balance components available for each product.
for column, name in [('adjustments_kbd', 'Adjustments'), ('net_receipts_kbd', 'Net regional receipts')]:
history(name, z[column])
if FUEL == 'crude':
history('Transfers', z.transfers_kbd)
history('Direct crude use', z.direct_use_kbd)
else:
history('Biofuel production', z.biofuels_kbd)
# Refinery operations can affect crude demand and gasoline output, so both
# notebooks receive lagged crude utilization and physical distillation capacity.
crude = national['crude'].reindex(z.index)
X['Crude refinery utilization — previous month'] = crude.utilization.shift(1)
X['Crude distillation capacity — previous month'] = crude.cdu_capacity_kbd.shift(1)
# Cross-product features test whether activity in the linked petroleum market
# contains information beyond the selected product's own recent history.
other_demand = 'Gasoline product supplied' if other_fuel == 'gasoline' else 'Crude refinery inputs'
other_production = 'Gasoline net production' if other_fuel == 'gasoline' else 'Crude production'
X[f'{other_demand} — previous month'] = other.demand_kbd.shift(1)
X[f'{other_production} — previous month'] = other.production_kbd.shift(1)
X[f'{other_fuel.title()} inventory change — previous month'] = other.change_mb.shift(1)
# Use one common sample for every comparison. This prevents a feature from looking
# better merely because it was evaluated on a different set of months.
keep = X.notna().all(axis=1) & targets.notna().all(axis=1)
X, targets = X.loc[keep], targets.loc[keep]
# Exclude March 2020 through March 2021 and any target whose lag window reaches
# that disruption. Offsets 0..13 cover the target month and the longest lag used.
eligible = pd.Series(True, index=X.index)
for offset in range(14):
dates = X.index - pd.DateOffset(months=offset)
eligible &= ~((dates >= pd.Timestamp('2020-03-01')) & (dates <= pd.Timestamp('2021-03-01')))
# Score the latest 72 eligible observations in six consecutive 12-month blocks.
# Every block is trained on all eligible observations strictly before that block.
scored_dates = X.index[eligible][-72:]
assert len(scored_dates) == 72
# A block can span a calendar gap because excluded COVID-related dates are absent.
folds = [pd.DatetimeIndex(d) for d in np.array_split(scored_dates, 6)]
features = [c for c in X if c not in calendar]
# Fail early if alignment, missing values, or duplicate feature names could make
# the ranking invalid or silently change the observations being compared.
assert X.index.is_monotonic_increasing and not X.index.has_duplicates
assert np.isfinite(X.to_numpy()).all() and np.isfinite(targets.to_numpy()).all()
assert len(features) == len(set(features))
periods = pd.DataFrame([{
'Period': i + 1, 'First scored month': dates.min().strftime('%Y-%m'),
'Last scored month': dates.max().strftime('%Y-%m'), 'Scored months': len(dates),
'Earlier training months': int(((X.index < dates.min()) & eligible).sum()),
} for i, dates in enumerate(folds)])
# Require a meaningful training history even for the earliest validation block.
assert periods['Earlier training months'].min() >= 36
# Keep regularization fixed so alpha is not tuned on the periods used to rank
# predictors. Standardization puts differently scaled inputs on comparable footing.
ALPHA = 10
def errors(y, columns):
"""Return out-of-sample absolute errors from expanding-window validation."""
blocks = []
for period, dates in enumerate(folds, 1):
# Training uses only eligible observations before the first scored month.
# This chronological split prevents future observations from leaking backward.
train = (X.index < dates.min()) & eligible
assert X.index[train].max() < dates.min()
model = make_pipeline(StandardScaler(), Ridge(alpha=ALPHA))
model.fit(X.loc[train, columns], y.loc[train])
prediction = model.predict(X.loc[dates, columns])
blocks.append(pd.DataFrame({
'period': period,
# Absolute error keeps the result in the target's thousand-b/d units.
'error': np.abs(y.loc[dates].to_numpy() - prediction),
}, index=dates))
return pd.concat(blocks)
rankings, calendar_importance = {}, []
for target_name in targets:
y = targets[target_name]
# These two reference models support the two definitions of importance:
# calendar-only for individual value, and all inputs for conditional value.
calendar_error = errors(y, calendar)
full_error = errors(y, list(X.columns))
rows = []
for feature in features:
# Individual test: add one feature to the calendar-only baseline.
added = errors(y, calendar + [feature])
# Conditional test: remove one feature from the full model and refit.
removed = errors(y, [c for c in X if c != feature])
# Positive gain means the feature reduces error beyond seasonality.
# Positive conditional value means the full model gets worse without it.
gain = calendar_error.error - added.error
conditional = removed.error - full_error.error
by_period = gain.groupby(calendar_error.period).mean()
rows.append({
'Predictor': feature,
'Error reduction vs calendar (thousand b/d)': gain.mean(),
'Error reduction vs calendar (%)': 100 * gain.mean() / calendar_error.error.mean(),
'Periods helped (of 6)': int((by_period > 0).sum()),
'Error increase when removed (thousand b/d)': conditional.mean(),
'Periods hurt by removal (of 6)': int((conditional.groupby(full_error.period).mean() > 0).sum()),
# Retain fold-level gains to show whether the average is historically stable.
**{f'Period {i} gain': by_period.loc[i] for i in range(1, 7)},
})
# The main rank is based on individual improvement over calendar seasonality.
table = pd.DataFrame(rows).sort_values(
'Error reduction vs calendar (thousand b/d)', ascending=False
).reset_index(drop=True)
table.index = table.index + 1
table.index.name = 'Rank'
rankings[target_name] = table
# Treat all 12 calendar indicators as one conceptual predictor when measuring
# how much seasonality contributes alongside every noncalendar feature.
without_calendar = errors(y, features)
calendar_importance.append({
'Target': target_name,
'Predictor': 'Calendar month (all 12 month indicators)',
'Error increase when removed (thousand b/d)': (without_calendar.error - full_error.error).mean(),
})
Most important variables¶
The top three individual predictors for each target appear first.
# Build a compact table from the full rankings. Only predictors that improve on
# the calendar baseline are eligible, and at most three are shown for each target.
gain_column = 'Error reduction vs calendar (thousand b/d)'
conditional_column = 'Error increase when removed (thousand b/d)'
summary = []
for target_name, table in rankings.items():
positive = table[table[gain_column] > 0].head(3)
for rank, row in positive.iterrows():
summary.append({
'Target': target_name,
'Rank': rank,
'Predictor': row.Predictor,
'Error reduction (thousand b/d)': row[gain_column],
'Periods helped (of 6)': row['Periods helped (of 6)'],
# This flag distinguishes a strong stand-alone predictor from one that
# still contributes after correlated inputs enter the full model.
'Adds information alongside other inputs': bool(row[conditional_column] > 0),
})
display(pd.DataFrame(summary).round(2))
| Target | Rank | Predictor | Error reduction (thousand b/d) | Periods helped (of 6) | Adds information alongside other inputs | |
|---|---|---|---|---|---|---|
| 0 | Supply: production + imports | 1 | Supply (production + imports) — previous 3-month average | 184.63 | 5 | True |
| 1 | Supply: production + imports | 2 | Crude distillation capacity — previous month | 178.07 | 5 | True |
| 2 | Supply: production + imports | 3 | Gasoline net production — previous 3-month average | 161.67 | 5 | True |
| 3 | Demand: gasoline product supplied | 1 | Gasoline product supplied — previous 3-month average | 93.92 | 6 | True |
| 4 | Demand: gasoline product supplied | 2 | Gasoline product supplied — previous month | 69.71 | 5 | True |
| 5 | Demand: gasoline product supplied | 3 | Inventory level — previous month | 36.00 | 3 | False |
For gasoline supply, the previous three month average of production plus imports was most important.
For gasoline demand, the previous three month average of product supplied was most important.
Individual predictor rankings¶
Supply: production + imports¶
# Plot and print the individual rankings for supply and demand separately.
target_name = 'Supply: production + imports'
table = rankings[target_name]
# Reverse the top 12 so the highest-ranked predictor appears at the top of barh.
top = table.head(12).iloc[::-1]
fig, ax = plt.subplots(figsize=(13, 7))
# Green bars improve on calendar; red bars increase average error.
colors = ['#237a57' if value > 0 else '#b65a52' for value in top[gain_column]]
ax.barh(top.Predictor, top[gain_column], color=colors)
ax.axvline(0, color='black', linewidth=0.8)
ax.set_title(FUEL.title() + ' ' + target_name.lower() + ': strongest individual predictors')
ax.set_xlabel('Reduction in average absolute error vs calendar month (thousand barrels/day)')
ax.set_ylabel('')
fig.tight_layout()
plt.show()
# Period-level columns are shown later; omit them here to keep this table readable.
display(table.drop(columns=[f'Period {i} gain' for i in range(1, 7)]).round(2))
| Predictor | Error reduction vs calendar (thousand b/d) | Error reduction vs calendar (%) | Periods helped (of 6) | Error increase when removed (thousand b/d) | Periods hurt by removal (of 6) | |
|---|---|---|---|---|---|---|
| Rank | ||||||
| 1 | Supply (production + imports) — previous 3-month average | 184.63 | 53.35 | 5 | 0.68 | 5 |
| 2 | Crude distillation capacity — previous month | 178.07 | 51.46 | 5 | 4.47 | 3 |
| 3 | Gasoline net production — previous 3-month average | 161.67 | 46.71 | 5 | 0.45 | 4 |
| 4 | Supply (production + imports) — previous month | 160.77 | 46.46 | 4 | 0.44 | 4 |
| 5 | Gasoline net production — previous month | 150.09 | 43.37 | 5 | 0.79 | 5 |
| 6 | Crude refinery inputs — previous month | 136.48 | 39.44 | 5 | -0.26 | 0 |
| 7 | Gasoline exports — previous month | 103.76 | 29.98 | 3 | 0.54 | 3 |
| 8 | Gasoline exports — previous 3-month average | 98.01 | 28.32 | 3 | -0.03 | 3 |
| 9 | Inventory level — previous month | 87.24 | 25.21 | 3 | 0.20 | 3 |
| 10 | Crude refinery utilization — previous month | 60.80 | 17.57 | 4 | -1.10 | 2 |
| 11 | Gasoline product supplied — previous 3-month average | 37.61 | 10.87 | 2 | 7.26 | 4 |
| 12 | Gasoline imports — previous 3-month average | 21.29 | 6.15 | 4 | 0.34 | 4 |
| 13 | Gasoline product supplied — previous month | 18.97 | 5.48 | 2 | 0.75 | 4 |
| 14 | Adjustments — previous 3-month average | 16.13 | 4.66 | 4 | 0.26 | 4 |
| 15 | Adjustments — previous month | 15.21 | 4.40 | 4 | 0.96 | 4 |
| 16 | Gasoline imports — previous month | 12.01 | 3.47 | 4 | 0.50 | 3 |
| 17 | Core balance (production + imports − exports − demand) — previous month | 1.08 | 0.31 | 3 | 0.02 | 3 |
| 18 | Core balance (production + imports − exports − demand) — previous 3-month average | 0.67 | 0.19 | 3 | 0.21 | 3 |
| 19 | Inventory change — previous month | 0.07 | 0.02 | 3 | -0.00 | 4 |
| 20 | Net regional receipts — previous month | 0.00 | 0.00 | 0 | 0.00 | 2 |
| 21 | Net regional receipts — previous 3-month average | 0.00 | 0.00 | 0 | 0.00 | 2 |
| 22 | Inventory level minus year-earlier level | -0.04 | -0.01 | 2 | 0.36 | 2 |
| 23 | Inventory change — previous 3-month average | -2.52 | -0.73 | 3 | 0.43 | 4 |
| 24 | Crude inventory change — previous month | -5.61 | -1.62 | 2 | 7.60 | 6 |
| 25 | Crude production — previous month | -23.87 | -6.90 | 2 | -1.46 | 2 |
| 26 | Biofuel production — previous 3-month average | -39.50 | -11.41 | 2 | -2.47 | 3 |
| 27 | Biofuel production — previous month | -41.11 | -11.88 | 2 | -1.10 | 2 |
Demand: gasoline product supplied¶
target_name = 'Demand: gasoline product supplied'
table = rankings[target_name]
# Reverse the top 12 so the highest-ranked predictor appears at the top of barh.
top = table.head(12).iloc[::-1]
fig, ax = plt.subplots(figsize=(13, 7))
# Green bars improve on calendar; red bars increase average error.
colors = ['#237a57' if value > 0 else '#b65a52' for value in top[gain_column]]
ax.barh(top.Predictor, top[gain_column], color=colors)
ax.axvline(0, color='black', linewidth=0.8)
ax.set_title(FUEL.title() + ' ' + target_name.lower() + ': strongest individual predictors')
ax.set_xlabel('Reduction in average absolute error vs calendar month (thousand barrels/day)')
ax.set_ylabel('')
fig.tight_layout()
plt.show()
# Period-level columns are shown later; omit them here to keep this table readable.
display(table.drop(columns=[f'Period {i} gain' for i in range(1, 7)]).round(2))
| Predictor | Error reduction vs calendar (thousand b/d) | Error reduction vs calendar (%) | Periods helped (of 6) | Error increase when removed (thousand b/d) | Periods hurt by removal (of 6) | |
|---|---|---|---|---|---|---|
| Rank | ||||||
| 1 | Gasoline product supplied — previous 3-month average | 93.92 | 46.40 | 6 | 9.18 | 6 |
| 2 | Gasoline product supplied — previous month | 69.71 | 34.44 | 5 | 0.80 | 5 |
| 3 | Inventory level — previous month | 36.00 | 17.78 | 3 | -1.27 | 4 |
| 4 | Supply (production + imports) — previous month | 34.98 | 17.28 | 2 | 0.09 | 5 |
| 5 | Supply (production + imports) — previous 3-month average | 32.65 | 16.13 | 2 | 0.79 | 6 |
| 6 | Crude distillation capacity — previous month | 23.54 | 11.63 | 2 | 3.73 | 3 |
| 7 | Gasoline net production — previous month | 20.54 | 10.15 | 2 | -0.00 | 4 |
| 8 | Gasoline net production — previous 3-month average | 19.48 | 9.63 | 2 | 0.77 | 5 |
| 9 | Crude refinery inputs — previous month | 10.04 | 4.96 | 2 | 0.22 | 3 |
| 10 | Inventory level minus year-earlier level | 7.46 | 3.69 | 4 | -0.77 | 3 |
| 11 | Core balance (production + imports − exports − demand) — previous month | 2.69 | 1.33 | 5 | 0.06 | 4 |
| 12 | Adjustments — previous month | 2.30 | 1.14 | 5 | 1.85 | 6 |
| 13 | Core balance (production + imports − exports − demand) — previous 3-month average | 2.08 | 1.03 | 6 | 0.03 | 3 |
| 14 | Inventory change — previous month | 1.87 | 0.93 | 5 | -0.03 | 3 |
| 15 | Adjustments — previous 3-month average | 1.51 | 0.75 | 5 | -0.39 | 2 |
| 16 | Inventory change — previous 3-month average | 1.10 | 0.55 | 4 | 0.19 | 4 |
| 17 | Net regional receipts — previous month | 0.00 | 0.00 | 1 | -0.00 | 0 |
| 18 | Net regional receipts — previous 3-month average | 0.00 | 0.00 | 1 | -0.00 | 0 |
| 19 | Crude inventory change — previous month | -0.34 | -0.17 | 2 | 6.02 | 6 |
| 20 | Gasoline imports — previous month | -1.03 | -0.51 | 2 | 0.70 | 6 |
| 21 | Gasoline imports — previous 3-month average | -3.83 | -1.89 | 1 | -0.32 | 2 |
| 22 | Crude refinery utilization — previous month | -5.86 | -2.90 | 2 | -0.90 | 3 |
| 23 | Gasoline exports — previous month | -13.91 | -6.87 | 2 | -0.14 | 2 |
| 24 | Gasoline exports — previous 3-month average | -14.31 | -7.07 | 2 | 1.43 | 4 |
| 25 | Biofuel production — previous month | -55.27 | -27.31 | 2 | -1.12 | 2 |
| 26 | Crude production — previous month | -56.05 | -27.69 | 2 | -2.41 | 4 |
| 27 | Biofuel production — previous 3-month average | -57.60 | -28.46 | 2 | -1.12 | 2 |
The three-month average of supply reduces error by about 185 thousand barrels/day versus calendar month alone.
For demand, the three-month average of product supplied reduces error by about 94 thousand barrels/day.
Which variables matter alongside the others?¶
These rankings remove one input at a time from a ridge model. Positive values mean that removing the input makes predictions worse. Negative values mean predictions improve without it.
# Reorder the same ranking tables by conditional importance: the change in
# full-model error when one predictor is removed and the model is refitted.
for target_name, table in rankings.items():
ordered = table.sort_values(conditional_column, ascending=False).head(12).iloc[::-1]
fig, ax = plt.subplots(figsize=(13, 7))
# Green means removal hurts predictions; red means removal improves them.
ax.barh(ordered.Predictor, ordered[conditional_column],
color=['#237a57' if v > 0 else '#b65a52' for v in ordered[conditional_column]])
ax.axvline(0, color='black', linewidth=0.8)
ax.set_title(target_name + ': which variables add information alongside all other inputs?')
ax.set_xlabel('Increase in average absolute error after removing and refitting (thousand barrels/day)')
fig.tight_layout()
plt.show()
# Calendar month is removed as a 12-column block, so report it separately from
# the one-column removal rankings plotted above.
display(pd.DataFrame(calendar_importance).round(2))
| Target | Predictor | Error increase when removed (thousand b/d) | |
|---|---|---|---|
| 0 | Supply: production + imports | Calendar month (all 12 month indicators) | 47.96 |
| 1 | Demand: gasoline product supplied | Calendar month (all 12 month indicators) | 39.84 |
The previous month's crude inventory change adds the most information in the removal test for gasoline supply, as removing it increases error by about 7.6 thousand barrels/day.
For gasoline demand, the three-month average of product supplied adds the most information, as removing it increases error by about 9.2 thousand barrels/day.
Crude inventory change is weak on its own, as above it did not reduce error for either target, but it adds information alongside the other inputs. Removing it also increases demand error by about 6.0 thousand barrels/day.
How consistent is each predictor?¶
Supply: production + imports¶
# Show the top ten individual predictors' gains within each validation block.
# This reveals whether a high overall average is broad or driven by one period.
target_name = 'Supply: production + imports'
table = rankings[target_name]
columns = [f'Period {i} gain' for i in range(1, 7)]
display(table.head(10).set_index('Predictor')[columns].round(2))
| Period 1 gain | Period 2 gain | Period 3 gain | Period 4 gain | Period 5 gain | Period 6 gain | |
|---|---|---|---|---|---|---|
| Predictor | ||||||
| Supply (production + imports) — previous 3-month average | 573.80 | 393.90 | 68.35 | 62.81 | 11.89 | -2.96 |
| Crude distillation capacity — previous month | 528.58 | 373.17 | 41.02 | 109.87 | -6.10 | 21.89 |
| Gasoline net production — previous 3-month average | 513.70 | 313.83 | 89.43 | 84.20 | 14.64 | -45.81 |
| Supply (production + imports) — previous month | 543.46 | 375.31 | 11.24 | 46.20 | -3.54 | -8.03 |
| Gasoline net production — previous month | 484.04 | 321.54 | 47.90 | 68.14 | 24.41 | -45.47 |
| Crude refinery inputs — previous month | 522.67 | 324.75 | 25.71 | 72.84 | 8.85 | -135.96 |
| Gasoline exports — previous month | 489.40 | 325.83 | -150.65 | 55.17 | -18.86 | -78.32 |
| Gasoline exports — previous 3-month average | 537.42 | 330.84 | -184.64 | 40.68 | -38.04 | -98.22 |
| Inventory level — previous month | 426.06 | 224.20 | -53.82 | 32.33 | -18.61 | -86.71 |
| Crude refinery utilization — previous month | 329.52 | 116.82 | -4.85 | 40.64 | 39.94 | -157.29 |
Demand: gasoline product supplied¶
target_name = 'Demand: gasoline product supplied'
table = rankings[target_name]
columns = [f'Period {i} gain' for i in range(1, 7)]
display(table.head(10).set_index('Predictor')[columns].round(2))
| Period 1 gain | Period 2 gain | Period 3 gain | Period 4 gain | Period 5 gain | Period 6 gain | |
|---|---|---|---|---|---|---|
| Predictor | ||||||
| Gasoline product supplied — previous 3-month average | 226.69 | 150.58 | 85.88 | 25.94 | 27.13 | 47.31 |
| Gasoline product supplied — previous month | 190.40 | 104.94 | 77.60 | -6.00 | 13.83 | 37.52 |
| Inventory level — previous month | 215.47 | 92.19 | 38.08 | -38.57 | -28.44 | -62.75 |
| Supply (production + imports) — previous month | 225.69 | 126.03 | -35.80 | -59.75 | -20.37 | -25.95 |
| Supply (production + imports) — previous 3-month average | 251.82 | 133.68 | -43.69 | -67.88 | -39.45 | -38.60 |
| Crude distillation capacity — previous month | 237.86 | 101.40 | -10.18 | -91.18 | -81.24 | -15.43 |
| Gasoline net production — previous month | 191.01 | 96.98 | -43.01 | -47.45 | -31.98 | -42.32 |
| Gasoline net production — previous 3-month average | 218.50 | 102.63 | -51.28 | -55.10 | -45.04 | -52.81 |
| Crude refinery inputs — previous month | 212.07 | 106.04 | -36.26 | -49.34 | -69.32 | -102.94 |
| Inventory level minus year-earlier level | 17.81 | -20.13 | 46.06 | -13.31 | 9.74 | 4.60 |
# Print the exact scored dates and the amount of earlier training data available
# for each block so the expanding-window design can be audited from the notebook.
display(periods)
| Period | First scored month | Last scored month | Scored months | Earlier training months | |
|---|---|---|---|---|---|
| 0 | 1 | 2018-05 | 2019-04 | 12 | 123 |
| 1 | 2 | 2019-05 | 2022-06 | 12 | 135 |
| 2 | 3 | 2022-07 | 2023-06 | 12 | 147 |
| 3 | 4 | 2023-07 | 2024-06 | 12 | 159 |
| 4 | 5 | 2024-07 | 2025-06 | 12 | 171 |
| 5 | 6 | 2025-07 | 2026-06 | 12 | 183 |
The three-month average of supply, crude distillation capacity and the three-month average of gasoline net production help predict supply in five of six periods when tested individually.
For demand, the three-month average of product supplied helps in all six periods. The previous month's product supplied helps in five periods, while inventory level helps in three.
Crude inventory change helps individually in only two periods for each target, but removing it from the full model makes predictions worse in all six periods for both supply and demand.
Conclusion¶
For gasoline supply, recent supply levels are the strongest individual predictors, with crude distillation capacity and gasoline net production as other strong predictors. For gasoline demand, recent product supplied is strongest. The three-month averages of supply and product supplied are stronger than their previous-month values.
Crude inventory change is weak on its own, but adds information alongside the other variables for both gasoline supply and demand. Gasoline inventory level is less consistent and does not improve the full demand model on average.
Comparing crude and gasoline¶
Gasoline is easier to predict compared to crude based on these tests as its errors are smaller both in barrels/day and relative to the size of supply or demand.
The strongest predictors are similar, but the variables that add information alongside the other inputs differ.
Table below shows average absolute error by the average target level to account for crude volumes being larger than gasoline volumes.
| Target | Full-model error, thousand barrels/day | Error as % of average target | Best individual predictor + calendar: error % |
|---|---|---|---|
| Crude supply | 354 | 1.82% | 1.81% |
| Gasoline supply | 138 | 1.40% | 1.63% |
| Crude demand | 338 | 2.07% | 1.57% |
| Gasoline demand | 110 | 1.22% | 1.20% |
Gasoline has lower relative errors for both targets, even when using just the strongest individual predictor plus calendar effects.
However, crude improves more relative to the baseline of only using the month of the year. Its full model reduces supply error by 89%, compared with 60% for gasoline. For demand, the reductions are 57% and 46%. This means crude has larger error reductions.
Do the important predictors differ?¶
The strongest individual predictor are the same.
| Target | Crude | Gasoline |
|---|---|---|
| Supply | Previous three-month average of production + imports | Previous three-month average of production + imports |
| Demand | Previous three-month average of refinery inputs | Previous three-month average of product supplied |
Second most important predictors were:
Crude supply: recent crude production
Crude demand: gasoline production
Gasoline supply: crude distillation capacity
Gasoline demand: recent product supplied
| Target | Largest error increase when a variable is removed |
|---|---|
| Crude supply | Crude year-over-year inventory difference: 17.1 thousand b/d |
| Crude demand | Crude year-over-year inventory difference: 19.4 thousand b/d |
| Gasoline supply | Previous-month crude inventory change: 7.6 thousand b/d |
| Gasoline demand | Three-month average gasoline product supplied: 9.2 thousand b/d |
Previous-month crude inventory change also helps the full gasoline demand model: removing it increases error by 6.0 thousand b/d.