Models and learning#

Every function each header declares, with the comment the header gives it. The header’s own opening comment is shown as written in the source.

ml_infer.h#

Source: main/ml_infer.h

On-board model: manoeuvre recogniser + estimator (ESP-DL), fed from the OBD poll loop.

Two models are embedded in flash and one runs at a time: CarMind E.V (the default) and, with CONFIG_CARMIND_REV_ENABLE,
CarMind R.E.V, about half the size, which also outputs its 64-value feature z for on-board fuel learning (rev_task.h).
The desktop app switches between them over USB (ml_select_model); the choice is kept in NVS (namespace "ml", key "model").

ml_feed() is called for every logged row and builds 1 s bins of the 10 pattern-layer channels.
ml_window_closed() is called when the pattern layer closes a 10 s window: it builds the model input from the last
ML_WINDOW seconds and hands it to the inference task through a message buffer, so the OBD task never waits on the
model. The inference task writes one line per window to sNNNNNN.ml.csv (keyed by window_seq, joins to the .pat).
If an input is still queued when the next one arrives, the new one is dropped and counted (soft real-time: the
deadline is the next window, a late answer is worth less than a fresh one later).

Function

Comment in the header

const char *ml_model_key(int id);

“ev”, “rev”

const char *ml_model_name(int id);

“CarMind E.V”, “CarMind R.E.V”

bool ml_model_available(int id);

built into this firmware

int ml_model_active(void);

the selected model; before ml_init the one a new board starts on

const char *ml_model_version(int id);

first 12 hex of the model file’s sha256, “” when not built in

const char *ml_model_target(int id);

the chip the model file was exported for (“esp32s3”, “esp32s31”)

int ml_selftest_state(void);

The loaded model’s self-test at load (CONFIG_CARMIND_ML_SELFTEST): 1 = its outputs on the embedded test input are within 1 LSB of ESP-PPQ’s PC simulation, 0 = further off, -1 = not run.

bool ml_select_model(int id, char *why, size_t why_n);

Switch models: unload the running one, load the other from flash, keep the choice in NVS. Writes NVS, so call it from a task with an internal-RAM stack (the USB door), and only while parked. If the new model does not load, the old one is loaded back and false is returned with the reason in why.

bool ml_rev_active(void);

R.E.V learning can run: R.E.V selected and loaded, with a 64-value “z” output, and not in safe mode.

void ml_set_rev_inputs(const ml_rev_in_t *in);

void ml_init(void);

void ml_feed(const obd_values_t *v, int64_t now_ms);

void ml_window_closed(uint32_t window_seq, int64_t start_ms, int64_t end_ms);

void ml_reset_history(void);

Resets the 1 s history, e.g. when a new adapter session starts.

void ml_drive_ended(void);

The drive ended: stop expecting model runs and drop the window history.

void ml_unload(void);

Free the model and load it back later (saves about 75 KB of PSRAM); safe while the task runs. Unused by default.

bool ml_load(void);

void ml_set_paused(bool paused);

Pause without freeing: feeding, windows and inference stop, the model stays resident (no reload, no allocation).

void ml_get_stats(ml_stats_t *out);

int ml_get_last_probs(float *out, int max);

Class probabilities of the last window, in ML_CLASSES order. Returns how many were written, 0 before the first window.

rev_learn.h#

Source: main/rev_learn.h

CarMind R.E.V: the learners that run on the board. Pure C, no ESP-IDF, so the host tests cover them (tests/host/test_rev_learn.c).

(a) Fuel head. The R.E.V network's 64-value feature z is fixed on the board; only a new linear read-out w (64 weights plus a
    bias, phi = [z, 1]) learns. It is fitted to this car's own fuel rate, with z computed as if MAF were missing, so that the
    estimate is ready for the moment MAF drops out. The fit is ridge regression with exponential forgetting and a pull toward
    the factory head w0:
        w* = argmin_w  sum_t lambda^(T-t) (y_t - phi_t.w)^2 + alpha |w - w0|^2
    whose normal equations are (A + alpha I) w* = b + alpha w0, with A = sum lambda^(T-t) phi phi^T and b = sum lambda^(T-t) phi y.
    Only A and b are kept (constant memory), plus a ring of the most recent REV_HOLDOUT pairs that is held out of A and b to judge
    a candidate: it replaces the live weights only if it beats them on that ring by REV_GATE and stays within REV_DRIFT times
    |w0| of w0; if the live weights fall REV_RESET behind the factory head on the ring, they go back to w0. Solved every
    REV_EVERY pairs during drives.
    Float throughout: the S3's FPU is single precision. With alpha = 80 on the diagonal, A + alpha I has a smallest eigenvalue of
    at least 80, so the float Cholesky stays far inside the 2 % gate; the host test runs a double solver beside it to check.
(b) Airflow constant. Absolute load (PID 0x43) is the air mass per intake stroke over the mass the cylinder holds at standard
    conditions, so  MAF [g/s] = k * absload [%] * rpm  with one constant per engine, k = 1.184 g/L * displacement / 12000.
    k is learned by scalar least squares with forgetting on 5 s means while MAF is reported, and used when MAF is missing.
    EXPERIMENTAL, untested on data: cars without PID 0x43 learn one k per rpm bin from calculated load (PID 0x04) instead.
(c) The source picker: running squared errors of the factory head, the learned head and the physics estimate against the same
    labels, so the board can tell which one to trust on this car when MAF is missing.
Labels are 5 s means of the measured fuel rate (PID 0x5E when the car reports it, else the MAF formula of econ.c), in L/h.

Function

Comment in the header

void rev_fuel_init(rev_fuel_t *L, const float *w0);

void rev_fuel_reset(rev_fuel_t *L);

Back to the factory head: weights, statistics, holdout and counters.

float rev_fuel_predict(const rev_fuel_t *L, const float phi[REV_NF]);

live head, standardised units

float rev_fuel_predict_prior(const rev_fuel_t *L, const float phi[REV_NF]);

factory head

bool rev_fuel_add(rev_fuel_t *L, const float phi[REV_NF], float y);

One labelled pair (y standardised). The oldest held-out pair moves into A and b with its Huber weight. Returns true when a commit is due. A pair with a value that is not finite, or with a residual over REV_REJECT_R times the running scale, is ignored.

rev_commit_t rev_fuel_commit(rev_fuel_t *L, char *msg, size_t msg_n);

Solve, gate, swap. msg (may be NULL) gets one human line.

float rev_fuel_drift(const rev_fuel_t *L);

|w_live - w0| / |w0|

void rev_label_reset(rev_label_t *r);

void rev_label_push(rev_label_t *r, int64_t t_ms, float lph);

bool rev_label_mean(const rev_label_t *r, int64_t end_ms, float *out);

Mean over (end - 5 s, end]: needs at least 2 rows spanning at least 2 s. Clipped to [0, 100] L/h.

int rev_phys_bin(float rpm);

void rev_phys_init(rev_phys_t *P);

void rev_phys_reset_k(rev_phys_t *P);

back to k0, statistics cleared

void rev_phys_new_drive(rev_phys_t *P);

drop the 5 s window, keep k

bool rev_phys_row(rev_phys_t *P, int64_t t_ms, bool abs_ok, float abs_load, bool load_ok, float load, bool rpm_ok, float rpm, bool maf_ok, float maf, float *maf_mean_est);

One row. abs_ok/abs_load (PID 0x43), load_ok/load (PID 0x04), rpm_ok/rpm, maf_ok/maf. Learns when MAF is present. Returns true and sets *maf_mean_est to k * (5 s mean of x) when the row fed the window (for scoring against labels).

bool rev_phys_estimate(const rev_phys_t *P, bool abs_ok, float abs_load, bool load_ok, float load, bool rpm_ok, float rpm, float *maf_gps, bool *ready);

Estimated MAF for this row, g/s, when MAF is missing. Absolute load uses k0 until it has learned; the experimental calculated-load path answers only once its bin has learned. ready (may be NULL) says whether k is learned.

bool rev_k_ready(const rev_k_t *k);

void rev_pick_init(rev_pick_t *p);

void rev_pick_score(rev_pick_t *p, float label, bool prior_ok, float prior, bool learned_ok, float learned, bool phys_ok, float phys);

One label against whichever estimates exist for the same 5 s (L/h).

rev_src_t rev_pick_choose(const rev_pick_t *p, bool phys_ready, bool learned_committed);

The lowest running error among the scored ones, ties to physics (one parameter, cannot overfit). Before enough scores: physics if k is learned, then the learned head if it has committed, then the factory head.

const char *rev_src_name(rev_src_t s);

rev_cal.h#

Source: main/rev_cal.h

Fill-up calibration of the board's fuel figures. Pure C, host-tested (tests/host/test_rev_learn.c).

One factor c such that c x (litres the board counted) = litres actually pumped. It multiplies every fuel source (car PID, MAF,
physics, learned and factory estimates), so the screen, the CSV and the trip totals show calibrated fuel; the learners keep
learning on raw values. With no fill-ups c = 1, which leaves every figure exactly as before.

The board keeps its own per-drive ledger (the last REV_CAL_SESSIONS drives): raw litres, km, and engine seconds with and without
a fuel rate. The user's fill-ups come from the desktop app. The tank-to-tank method:
  - A full fill (to the pump's automatic click-off) closes an interval that started at the previous full fill; partial fills in
    between add their litres. Interval i: P_i = litres pumped, R_i = raw litres the board counted over the drives in between.
  - An interval counts only if R_i >= 5 L, fuel-rate coverage >= 95 % of engine time, all its drives are still in the ledger,
    and 0.5 <= P_i / R_i <= 2 (a typo such as 400 L fails).
  - A trip-computer figure (L/100 km over a range of drives) gives P = L/100km x km / 100 and counts half as much as a pump interval.
  - c = sum_i g^(n-i) w_i P_i / sum_i g^(n-i) w_i R_i over the accepted intervals, newest weighted most (g = 0.7 per interval),
    clamped to [0.7, 1.4]. A ratio of sums, so a short top-up interval cannot swing c. The first full fill only sets the start.

Function

Comment in the header

void rev_cal_init(rev_cal_t *c);

void rev_cal_row(rev_cal_t *c, uint32_t session, float dt_s, float raw_lph, float kph, bool engine_running);

One OBD row: raw fuel rate (negative = unknown), speed (negative = unknown), engine running (rpm > 0), seconds since the last row of this drive. Gaps over 10 s are not counted. cal is the factor in force for the lifetime totals.

bool rev_cal_set_fills(rev_cal_t *c, const rev_fill_t *fills, int n, uint32_t fill_rev);

Replace the fill-up list (any order; at most REV_CAL_FILLUPS) and recompute c. Returns false and changes nothing if an entry is not plausible (litres 0..300, L/100 km 1..60, a trip range backwards).

float rev_cal_compute(rev_cal_t *c);

Recompute c, used, skipped and the interval list from the fill-ups and the ledger.

void rev_cal_reset(rev_cal_t *c);

Forget the fill-ups and go back to c = 1 (the ledger and lifetime totals stay).

const char *rev_iv_why_name(int why);

rev_tank.h#

Source: main/rev_tank.h

CarMind R.E.V tank level learner. Pure C, no ESP-IDF, host-tested (tests/host/test_rev_tank.c).

Learns C_R, the tank's capacity in RAW counted litres per 100 % of the fuel level sender (PID 0x2F), from the level against
the litres the board counts (the same raw litres rev_cal_row() integrates, before the fill-up factor c). True litres are
c x raw litres, so the true capacity is c x C_R. Range needs no c: raw litres left / raw litres per km.

Model between fill-ups: level = a_k - (100 / C_R) x raw_litres + noise, one intercept a_k per segment. Each segment keeps
Welford sums of (x = raw litres, y = level); closed segments fold into pooled sums with forgetting per litre; the slope is
beta = -Pxy / Pxx and C_R = 100 / beta. Intercepts drop out, so level changes while the engine is off never enter the slope.

Fill-ups: at each restart (new drive, or engine off >= 60 s) the mean of the first K level samples is compared with the
mean of the last K before the stop. A jump >= REV_TANK_FILL % is a fill-up SUGGESTION for the desktop app. Suggestions never
feed rev_cal's c: their litres are computed from c and C_R, so feeding them back would make c a function of itself.

Function

Comment in the header

void rev_tank_init(rev_tank_t *t);

void rev_tank_reset(rev_tank_t *t);

Forget everything learned (new car, sender replaced).

void rev_tank_restart(rev_tank_t *t);

Start of a drive, and after rev_state_load(): the next K level samples are judged against the last K.

void rev_tank_row(rev_tank_t *t, float dt_s, float raw_lph, float kph, bool engine_running);

Every OBD row: seconds since the last row, raw fuel rate (negative = unknown), speed (negative = unknown), engine running.

void rev_tank_level(rev_tank_t *t, uint32_t session, float level_pct);

Only on rows where PID 0x2F was polled and answered: the level in %. session is the current session id (for suggestions).

void rev_tank_get(const rev_tank_t *t, rev_tank_out_t *o);

void rev_tank_ack(rev_tank_t *t, uint32_t seq);

The app has taken the suggestions up to and including seq.

bool rev_tank_valid(const rev_tank_t *t);

Finite-value check for the loader.

rev_torque.h#

Source: main/rev_torque.h

CarMind R.E.V: engine (brake) torque from physics, for cars that do not report torque. Pure C, host-tested
(tests/host/test_rev_learn.c).

Brake torque = indicated torque from the fuel's energy minus friction:
    T_b = eta_i * mdot_f * Q_LHV / omega  -  T_f,      omega = 2 pi rpm / 60
    T_f = (rubbing MEP + pumping MEP) * V_d / (4 pi)    (four-stroke: one cycle every 4 pi radians)
    rubbing MEP = 97 + 15 N + 5 N^2 kPa, N = rpm / 1000;  pumping MEP = max(p_amb - MAP, 0)
mdot_f comes from the car's fuel rate (PID 0x5E), else MAF / AFR, else the learned airflow constant (k * absolute load
* rpm / AFR). The engine size V_d = 12000 k / 1.184 L comes from the same learned k, so the estimate scales with the
engine of the car it runs in. In a decel fuel cut mdot_f = 0 and T_b = -T_f (engine braking).
A car that reports actual torque (PID 0x62) and reference torque (0x63) is used directly instead.
What learns per car: k (and so V_d), in rev_phys_row. eta_i is a prior; nothing on the board can learn it without a
torque label. Petrol only.

Function

Comment in the header

void rev_tq_init(rev_tq_t *T);

void rev_tq_new_drive(rev_tq_t *T);

drop the 5 s window, keep the PID convention

float rev_tq_vd(const rev_phys_t *P);

12000 k / 1.184 L, clamped; from k0 until k has learned

float rev_tq_friction(float rpm, float map_kpa, bool map_ok, float baro_kpa, float vd_l);

float rev_tq_fuel(float mdot_kgps, float rpm);

eta Q mdot / omega; 0 when not running

rev_cut_t rev_tq_cut(const rev_tq_in_t *in);

bool rev_tq_row(rev_tq_t *T, const rev_tq_in_t *in, const rev_phys_t *P, rev_tq_row_t *out);

One row: picks the source, applies the fuel cut, writes out. false when nothing usable.

void rev_tq_push(rev_tq_t *T, int64_t t_ms, const rev_tq_row_t *row, float kph);

bool rev_tq_mean(const rev_tq_t *T, int64_t end_ms, float *t_b, rev_tq_src_t *src, bool *show);

Mean over (end - 5 s, end]: at least 2 rows spanning at least 3 s. show = false while the mean speed is under 2 km/h (at a standstill in Drive the model reads high: torque-converter drag is not in it).

const char *rev_tq_src_name(rev_tq_src_t s);

rev_state.h#

Source: main/rev_state.h

The R.E.V learned state on the SD card (/sdcard/obd/rev_state.bin). Pure C, host-tested.

Layout, little-endian, version 1:
  header  { magic "REV1", u16 version, u16 nf, u32 prior_id, u32 sizes of the fuel, physics, picker and calibration sections }
  fuel    w_live[nf], A[nf][nf] (upper triangle used), b[nf], commits, resets, kept, failed, learned   (only the statistics)
  physics rev_phys_t k values and sums (the 5 s window is not saved)
  picker  rev_pick_t
  cal     rev_cal_t (ledger, fill-ups, c, lifetime totals)
  u32 CRC-32 (IEEE) of everything before it
The sections are written as the firmware's own structs, so a change to one of them changes its size and the file is
started fresh rather than misread. prior_id is the first 4 bytes of the R.E.V model's sha256: a new model has a new z
space, so only the fuel head is dropped; physics, picker and calibration do not depend on z and are kept.

Save: write path.tmp, flush, fsync, remove path, rename (FAT's rename does not replace). Load: path, else path.tmp (a power
cut between remove and rename leaves only the .tmp), each checked for magic, version, sizes, CRC and finite values.

Function

Comment in the header

uint32_t rev_crc32(uint32_t crc, const void *data, size_t n);

bool rev_state_save(const char *path, const rev_fuel_t *fuel, const rev_phys_t *phys, const rev_pick_t *pick, const rev_cal_t *cal, uint32_t prior_id);

fuel may be NULL (no head to save).

rev_load_t rev_state_load(const char *path, rev_fuel_t *fuel, rev_phys_t *phys, rev_pick_t *pick, rev_cal_t *cal, uint32_t prior_id);

On REV_LOAD_NONE nothing is changed. On REV_LOAD_OTHER_MODEL the fuel head is left as it was (the caller’s prior). The fuel head’s w0 pointer is kept; fuel may be NULL to skip it.

bool rev_blob_save(const char *path, uint32_t magic, const void *data, uint32_t size);

A small learner’s state in a file of its own (the tank learner: /sdcard/obd/rev_tank.bin), so adding a learner does not change rev_state.bin’s layout. Layout: u32 magic, u32 size, the struct, u32 CRC-32. Same tmp + fsync + rename on save; on load path, else path.tmp, each checked for magic, size and CRC. Returns false and leaves *data unchanged on any failure.

bool rev_blob_load(const char *path, uint32_t magic, void *data, uint32_t size);