It's askthegrid.com - always interested in the performance of the grid, especially as I drive by pieces of the infrastructure in the real world. Built this to ask questions and it works quite well. It's free, please see if you can stump it!
We just open-sourced ForecastOps, a local-first Python library we built for our own forecasting workflows, including both human-created and agent-created forecasting programs. It captures forecast runs from existing code, validates and scores them, stores artifacts locally as Parquet with DuckDB indexing, and provides a local UI for residuals, benchmarks, backtests, groups, and horizon/regime slices. I’d love feedback from data engineers on the architecture, storage model, and whether this fits real forecasting/data workflows.
We just opensourced ForecastOps, a local first py library for evaluating and observing forecasting workflows.
We've been using an early version of it internally, both human and agent made forecasting programs were producing lots of forecast runs, and we needed a lightweight way to capture, validate, score, group, and inspect them without shipping raw forecast data to a hosted service.
It sits alongside existing forecasting code and stores forecast artifacts locally as Parquet, with runs/metrics indexed in DuckDB. It includes validation, residuals, benchmark skill, rolling-origin backtests, run groups, horizon/regime slices, and a local UI.
It does not train models or upload data. Optional otel metrics/traces can be routed to tools like Datadog while raw artifacts stay local.
I’d love feedback from data engineers on the architecture, storage model, and where this would or would not fit into real forecasting/data workflows. I'd love to shape this into an "ops" style project - there are great MLOps and LLMOps things out there, but nothing perfect for this...
We just opensourced ForecastOps, a local first py library for evaluating and observing forecasting workflows.
We've been using an early version of it internally, both human and agent made forecasting programs were producing lots of forecast runs, and we needed a lightweight way to capture, validate, score, group, and inspect them without shipping raw forecast data to a hosted service.
It sits alongside existing forecasting code and stores forecast artifacts locally as Parquet, with runs/metrics indexed in DuckDB. It includes validation, residuals, benchmark skill, rolling-origin backtests, run groups, horizon/regime slices, and a local UI.
It does not train models or upload data. Optional otel metrics/traces can be routed to tools like Datadog while raw artifacts stay local.
I’d love feedback from data engineers on the architecture, storage model, and where this would or would not fit into real forecasting/data workflows. I'd love to shape this into an "ops" style project - there are great MLOps and LLMOps things out there, but nothing perfect for this...
We just open-sourced mlx-chronos2, a pure MLX inference backend for Amazon's Chronos-2 time series foundation model. No PyTorch required for inference.
Install: pip install mlx-chronos2
Repo: https://github.com/tsfm-ai/chronos2-mlx
| Config | PyTorch CPU | MLX int8+compile | Speedup |
|---|---|---|---|
| Batch=1, ctx=128 | 81 ms | 30 ms | 2.7× |
| Batch=8, ctx=128 (per series) | 51 ms | 6 ms | 64× |
| Batch=8, ctx=512 | 140 ms | 14 ms | 10× |
This tripped me up at first. Metal GPU kernel dispatch costs ~1ms per launch. Chronos-2 has ~120 matmuls per forward pass. At batch=1 with a short context, you're paying 120ms in dispatch overhead before any actual compute runs. PyTorch on Apple Silicon uses Accelerate/AMX (CPU) with near-zero dispatch overhead — so it wins at small batch.
mx.compile() fuses those 120 dispatches into a single Metal command buffer. That's the 2.7× at batch=1.
At batch=8, GPU parallelism kicks in and PyTorch's linear CPU scaling gets crushed. Hence the 64× per-series number.
Practical rule: compile for batch ≤ 4 or context ≤ 256. Skip it for large batch + long context.
| Mode | MAE | Δ vs fp32 | 90% Coverage | Memory |
|---|---|---|---|---|
| fp32 | 2.707 | — | 89.7% | 480 MB |
| int8, gs=64 | 2.706 | −0.03% | 89.5% | 120 MB |
| int4, gs=64 | 2.740 | +1.23% | 85.7% | 60 MB |
int8 is essentially free. 4× memory reduction, no measurable accuracy loss. int4 degrades interval coverage noticeably (85.7% vs 89.7%), which matters if you're using the quantile outputs for risk decisions.
.npz)pipe.embed() — REG token pooling for series-level similaritymx.compile compatibleQuestions welcome. Accuracy eval methodology is in the blog post if you want to reproduce it.
Full writeup with charts: https://tsfm.ai/blog/chronos-2-apple-silicon-mlx
We put up a comparison looking at TSFM research growth over the past few years and how it stacks against early LLM development: publication volume, model release tempo, open weights, benchmarks, and the missing ChatGPT catalyst for forecasting.
current read is that TSFMs look mostly pre chatgpt in some ways, but the category is also structurally different: forecasting has harder evals (but more structured), less obvious UX, and value may come less from fluent interaction and more from confidence + forecast, representation learning, and downstream decision-making.
Curious how people here think about it, especially if current models like Toto 2, Chronos, et are meaningfully changing production forecasting yet, or whether we’re still mostly in benchmark/model-zoo territory: https://tsfm.ai/blog/tsfm-vs-llm-research-compared
Specifically, Toto 2 looks exciting as it is clearly scaling well.
