Live Table Telemetry From A Persistent Q-Learning Worker

A blackjack agent runs continuously on the same machine as this portfolio site. The browser polls the local API, and the dashboard below reflects the live table state, bankroll, and latest decision trace in near real time.

Connecting... Mode: pending Waiting for worker state. /api/blackjack/state

Agent

Table Phase

Idle

Current Prompt

Waiting for prompt

Cash

Most Cash

Win Rate

0.0000

Time At Table

Rounds

Hands

Wins / House / Draws

0 / 0 / 0

Current Bet

Dealer Visible

Shoe Remaining

Action States

Bet States

Player Busts

Dealer Busts

Double Downs

Splits

> Waiting for live state...

Blackjack, But Running Live

This project started as a blackjack simulator and eventually turned into a persistent Q-learning experiment. The action policy learned how to hit, stay, double down, and split, and a separate betting policy learned how aggressive to be before the cards were dealt.

The live page on the left is not a mock-up. A Python worker keeps playing in the background on the same machine that serves this site, while the browser polls a lightweight local API for the latest bankroll, hand state, and decision log. That setup made the deployment much cleaner than trying to scrape terminal output or juggle file snapshots.

I kept the environment intentionally small enough to reason about, but still gave it the parts that make blackjack interesting: a finite shoe, split rules, double-down rules, a running bankroll, and session stats that persist as the worker keeps going. The action policy plateaued around the kind of hand win-rate I expected, so the next interesting knob to learn was betting.

The betting policy uses pre-deal information only: available cash, shoe depth, and a simple count-style signal that measures whether the shoe is becoming high-card heavy or low-card heavy. That way the agent can behave a little more like a real table player and size bets based on advantage instead of always wagering a flat amount.

Current Architecture

The blackjack engine owns the shoe, round state, split logic, payouts, and event feed.
The Q player carries separate action and bet Q-tables saved under checkpoints/.
The live service exposes /api/blackjack/state for this page to poll.

Future pass: expose the current state's action values and animate the best Q-value in real time so the dashboard can show which move the model thinks is strongest before it commits.

Source

github.com/jcs101194/ml_blackjack