Run #6 success proof_build_manual

Run detail

Monitor the academic pipeline and ML actions from one page.

Open Grafana Back to batch All runs

Started

2026-03-06 22:20

Finished

2026-03-06 22:21

Assets processed

Days processed

1742

Operator guide

Model run #6 is available. Next actions: Validate, Backtest, then Forecast.

Stage flow

Each stage shows its summary, errors, and payload details.

Load

pending

Waiting

Cycles / Baseline / FSI

pending

Waiting

Features / TFT Dataset

pending

Waiting

ML Train

pending

Waiting

Validate

pending

Waiting

Backtest

pending

Waiting

Forecast

pending

Waiting

Run parameters

mode	proof_build_manual
source	ui
batch_id	5

Recent events

No events yet.

Model runs

ID	Status	Checkpoint
#6	success	/app/models/run_6_modelrun_6/best.ckpt
#5	success	/app/models/run_6_modelrun_5/best.ckpt
#4	success	/app/models/run_6_modelrun_4/best.ckpt
#3	success	/app/models/run_6_modelrun_3/best.ckpt
#2	success	/app/models/run_6_modelrun_2/best.ckpt

Latest model details

Model run	#6
Status	success
Checkpoint	/app/models/run_6_modelrun_6/best.ckpt

Training params

dropout	0.1
val_days	90
batch_size	64
max_epochs	25
hidden_size	64
feature_spec	v1_log1p_fsi
learning_rate	0.001
attention_heads	4
target_transform	log1p
max_encoder_length	21
max_prediction_length	14

Training metrics

rows	1478
backtest	{'rows': 1478, 'assets': 3, 'h1_mae': 0.0, 'h7_mae': 0.025994809741441745, 'h14_mae': 0.03669440636733416, 'h1_rmse': 0.0, 'h7_rmse': 0.02600044526430903, 'h14_rmse': 0.03670950220802719, 'h1_smape': 0.0, 'h7_smape': 200.0, 'h14_smape': 200.0, 'used_cutoffs': [957, 964, 971], 'skipped_cutoffs': [{'cutoff': 978, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 985, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 992, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 999, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 1006, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 1013, 'reason': 'ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required by StandardScaler.'}, {'cutoff': 1020, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}, {'cutoff': 1027, 'reason': 'assertion_error: filters should not remove entries all entries - check encoder/decoder lengths and lags'}], 'num_used_cutoffs': 3, 'requested_cutoffs': [957, 964, 971, 978, 985, 992, 999, 1006, 1013, 1020, 1027], 'negative_pred_rate': 0.3333333333333333, 'num_prediction_rows': 9, 'num_skipped_cutoffs': 8, 'num_requested_cutoffs': 11}
qa_summary	{'rows': 1478, 'assets': 3, 'by_asset': [{'rows': 358, 'fsi_max': 1.4368447795979469, 'fsi_min': 0.0, 'time_max': 524, 'time_min': 0, 'asset_code': 'E102A', 'neg_raw_fsi': 0}, {'rows': 740, 'fsi_max': 5.04221501862956, 'fsi_min': 0.0, 'time_max': 1047, 'time_min': 0, 'asset_code': 'E102B', 'neg_raw_fsi': 0}, {'rows': 380, 'fsi_max': 5.04221501862956, 'fsi_min': 0.0, 'time_max': 675, 'time_min': 0, 'asset_code': 'E102C', 'neg_raw_fsi': 0}], 'time_idx_max': 1047, 'time_idx_min': 0, 'top_null_counts': {'day': 0, 'fsi': 0, 'flow': 0, 'day_cos': 0, 'day_sin': 0, 'delta_p': 0, 'baseline': 0, 'cycle_id': 0, 'time_idx': 0, 'is_weekend': 0, 'cleaning_event': 0, 'pdi_normalized': 0, 'steam_feed_ratio': 0, 'system_t_skin_avg': 0, 'system_t_skin_max': 0, 'total_vc5_rate_m3h': 0, 'days_since_cleaning': 0, 'furnaces_in_service': 0, 'total_ngl_rate_km3h': 0, 'pws_feedwater_rate_m3h': 0}, 'insufficient_assets': [], 'negative_target_count': 0, 'negative_raw_fsi_count': 0}
max_time_idx	1047
cutoff_time_idx	957
best_model_score	0.06145723536610603
target_transform	log1p
zero_target_count	741
negative_raw_fsi_count	0

Validation metrics

Horizon	Metric	Value
1	mae	0.0
1	rmse	0.0
1	smape	0.0
7	mae	0.025994809741441745
7	rmse	0.02600044526430903
7	smape	200.0
14	mae	0.03669440636733416
14	rmse	0.03670950220802719
14	smape	200.0

Physics metrics

Metric	Value
actual_fsi_max	1.4368447795979469
actual_fsi_max	5.04221501862956
actual_fsi_max	5.04221501862956
actual_fsi_mean	0.21455832557829554
actual_fsi_mean	0.13704504075908946
actual_fsi_mean	0.1616484376933917
negative_pred_rate	0.3333333333333333
rows_in_eval_df	740.0
rows_in_eval_df	358.0
rows_in_eval_df	380.0