The Bitter Lesson and How We Represent Actions
My understanding of The Bitter Lesson began to shift when I realized something:
My dad had not exactly started from scratch when he beat me at Go.
I first read the essay in the first year of my PhD, shortly after moving from “traditional robotics” into robot learning. It was remarkably short, but it changed how I thought about the field. Many things we had worked so hard to design might eventually be surpassed by a more general approach that simply learned from enough experience and computation.
The part about Go brought back a childhood memory.
I took Go lessons for two years in primary school. Among my classmates, I was practically unbeatable. Then I taught my dad the rules. Before long, he was beating me convincingly.
I had spent more time studying Go. My teacher had taught me capture techniques, opening patterns, and familiar sequences. My dad knew none of them. Somehow, this self-taught player had demolished my little expert system, carefully packed with human knowledge.
Devastated, I retired.
So when I read The Bitter Lesson years later, my first reaction was: absolutely. I had personally experienced the bitterness in primary school.
Sutton’s argument is about the long run: general methods that can keep using more computation, especially search and learning, tend to outlast approaches built around carefully encoded domain knowledge. 1 It made me wonder how much of our design work was necessary, and how much came from being uncomfortable letting a model learn.
Then, much later, I noticed the missing part of my childhood story.
My dad had never studied those opening patterns. But I had taught him the rules of the game: which moves were legal, how stones were captured, and what counted as winning. Even AlphaGo Zero, which learned without human game records, received a well-formulated problem. It did not first have to inspect a camera feed and discover which moves were legal and which were not. 2
I had not taught my dad how to win. I had already told him what game he was playing.
He could focus on how to play well, because he already knew the rules and what he was trying to achieve.
This story does not refute Sutton’s argument. It does, however, help me distinguish prescribing a solution from choosing how to express a problem.
If a representation makes useful relationships in the data easier to learn, I see no contradiction in studying it while believing in learning.
Letting the model find the answer does not make the way we write the question irrelevant.
This reminds me of a game from my university orientation: ‘How many mosquitoes?’
Someone who knows the rule makes a few gestures, then suddenly asks how many mosquitoes there are. I was completely lost. Was I supposed to count fingers? Claps? Watch which hand moved first?
I did some very serious feature engineering. I also kept getting the answer wrong.
Eventually, someone told me the rule. Everything made sense immediately, and I acquired a new way to annoy other people.
What stayed with me was how many examples I had seen while paying attention to the wrong things. Could I have figured it out alone? Probably, with enough observations and experiments. But the space of possible rules was large, and searching through it without knowing which clues mattered was costly and inefficient.
Giving me the rule gave away the answer. That is not what I want to do for a robot. The question I care about is whether we can make useful relationships easier to see while leaving the behavior itself to be learned.
Data, compute, and robot time all cost something. A robot still has to gain a substantial amount of experience in the physical world. It seems reasonable to ask whether we have accidentally made its learning problem harder than it needs to be.
What should an action token represent?
That question led to our paper on Direction–Scale Decomposition (DSD): before deciding how to tokenize actions, what exactly are we asking the tokenizer to encode? 3
A vision-language-action model predicts robot actions from observations and instructions. In a common discrete-token setup, each action describes a small change in the end effector’s position and orientation, together with a gripper command. These numerical increments are normalized and converted into tokens.
Imagine two demonstrations following the same path at different speeds. With a fixed sampling rate, the faster demonstration has larger displacements at each step. Coordinate-wise binning can therefore produce different action tokens even when the direction is unchanged.
Across datasets, different normalization statistics can also map the same physical motion to different tokens. If the normalization used in training and execution differs, it can even change the direction of the decoded motion. 3
I picture the robot trying to learn a task while also wondering: ‘Are these movements different, or was this teleoperator in a hurry?’
The model has to learn which token patterns describe related motions while also learning the task. The representation determines how readily those relationships are available.
Speed and magnitude matter, so the representation needs to retain them. It can also make shared motion structure explicit.
DSD separates a translation into its direction and distance, and a relative rotation into its axis and angle. Magnitudes stay in separate scale channels, and the gripper command is unchanged. The decomposition happens before tokenization, so it can be studied with different tokenizers. 34
We have separated ‘which way’ from ‘how far.’ The model still has to learn both.
For a fixed coordinate convention, a longer displacement along the same direction changes the scale without changing the direction target. The geometric information was already in the original action; DSD makes this structure explicit while preserving the represented motion before quantization.
What changes in practice?
We evaluated DSD with two tokenizers: uniform binning (BIN) and the B-spline tokenizer BEAST. Each pair uses the same model backbone, allowing us to study the effect of changing the representation supplied to the tokenizer. 3
| Evaluation / tokenizer | Baseline | With DSD | Gain (pp) |
|---|---|---|---|
| LIBERO BIN · average | 88.5% | 92.3% | +3.8 |
| LIBERO BEAST · average | 88.0% | 91.9% | +3.9 |
| SimplerEnv BIN · Bridge only | 40.0% | 40.7% | +0.7 |
| SimplerEnv BIN · five-dataset mixture | 33.0% | 43.3% | +10.3 |
| Real robot BIN · without robotics pretraining | 60.0% | 73.3% | +13.3 |
| Real robot BIN · with robotics pretraining | 65.0% | 76.7% | +11.7 |
The mixed-dataset result is the one that most makes me think about The Bitter Lesson. In SimplerEnv, adding four datasets to Bridge training reduced BIN’s overall success from 40.0% to 33.0%, while DSD-BIN increased from 40.7% to 43.3%. The gap between the two representations grew from 0.7 to 10.3 percentage points. 3
The gains vary by task, and larger mixtures remain to be tested. Within this five-dataset comparison, however, the action representation changed whether adding heterogeneous data helped or hurt overall performance. That makes representation relevant to the question of scale.
We often ask how much more data a model needs. These results also make me ask how much of the available data it can use effectively, given the way its actions are represented.
ResNet offers a useful analogy. Residual connections made deeper networks easier to optimize while leaving the visual concepts to be learned. 5 The broader idea is that a design choice can help a general learning method make better use of its capacity. DSD explores that possibility at the level of action representation.
What has changed for me since first reading The Bitter Lesson is the question I ask about a design: does it help the learner discover useful relationships in its experience, and does that help persist as we give it more data and computation?
Back when I taught my dad Go, I explained the rules of the game. I still left him to figure out his next move.
He learned that part himself.
Then he used it to beat me.
The paper gives the representation, training details, and full experimental comparisons; the project page shows the policies in action.
Research notes · October 27, 2026
Further experiments and observations
The paper leaves several practical questions worth examining more closely. The experiments below explore normalization, direction-loss weighting, and a possible explanation for the gains.
How much do the normalization bounds matter?
Our BIN baseline follows the OpenVLA convention: normalize each action coordinate using its 1st and 99th percentiles, q01 and q99, then clip and discretize it. 6
The effect of the bounds varies by setting. In its normalization ablation, LingBot-VLA 2.0, Section 6.1, reports nearly identical average success rates for min–max and q01–q99 normalization: 47.5% and 47.4%. 7
We thank Poke & Wiggle for independently reproducing our work and investigating these implementation choices. Their evaluation found a benefit for DSD over BIN with q01–q99 clipping, while a wider 0.1st–99.9th-percentile BIN baseline performed on par with their DSD implementation. Their report also documents differences in the controller interface and direction-loss weighting. 8
Motivated by that result, we tested min–max bounds for BIN on cube stacking. Widening the range improved success in our setup, with DSD-BIN retaining the higher success rate.
| Condition | Successes | Success rate |
|---|---|---|
| BIN · q01–q99 Paper baseline | 12 / 30 | 40.0% |
| BIN · min–max New ablation | 15 / 30 | 50.0% |
| DSD-BIN Paper result | 19 / 30 | 63.3% |
The observed improvement from widening BIN’s bounds is 10 percentage points. It narrows the gap to DSD-BIN from 23.3 to 13.3 points.
Our practical takeaway is to choose the normalization range together with the controller and action definition. Clipping can remove useful corrective motions, while a wider range spreads a fixed number of bins over a larger interval. Control frequency, dead zones, and whether increments are executed directly or accumulated into position targets can affect that trade-off.
Step weighting or a soft ramp?
When a motion is very small, a small perturbation can produce a large change in its direction target. The implementation used for our paper therefore down-weights the direction-token loss with a step: w(s) = 0.1 when the motion scale s is below a threshold τ, and w(s) = 1 otherwise. 3
In Poke & Wiggle’s evaluation, DSD used a soft ramp, w(s) = clip(s / τ, 0.1, 1). 8 We evaluated this weighting rule on LIBERO and compared it with both BIN and the paper’s step-weighted DSD-BIN.
Scroll horizontally to compare all columns.
| Suite | BIN (paper) | DSD-BIN Step (paper) | DSD-BIN Soft ramp (ablation) | Soft − step (pp) |
|---|---|---|---|---|
| Spatial | 83.4% | 93.6%+10.2 pp vs BIN | 91.6%+8.2 pp vs BIN | −2.0 |
| Object | 93.4% | 96.8%+3.4 pp vs BIN | 96.2%+2.8 pp vs BIN | −0.6 |
| Goal | 91.8% | 95.0%+3.2 pp vs BIN | 83.0%−8.8 pp vs BIN | −12.0 |
| Long / LIBERO-10 | 85.2% | 83.8%−1.4 pp vs BIN | 87.6%+2.4 pp vs BIN | +3.8 |
| Average | 88.5% | 92.3%+3.8 pp vs BIN | 89.6%+1.1 pp vs BIN | −2.7 |
The soft ramp performs better on LIBERO-10, but its four-suite average is 2.7 percentage points below the paper’s step weighting, with the largest decrease on Goal. Compared with BIN’s 88.5% average, step-weighted DSD-BIN gains 3.8 points and the soft-ramp variant gains 1.1 points. The weighting rule therefore matters when comparing and reproducing these results.
Making sense of the gains
Changing the action representation alone can lead to a substantial improvement. The size of the gain surprised us, so we tried to understand why.
With coordinate-wise binning, demonstrations following the same path at different speeds produce different token patterns. They may form separate modes in the action targets even when they express the same underlying behavioral choice. Our hypothesis is that this makes the prediction problem more complicated than the choice of behavior requires.
This may connect to the factorization error described in When Do Discrete Actions Make Good Continuous Controllers?. When action tokens are predicted independently, coordinates or time steps from incompatible modes can be combined into an incoherent action chunk. 9 In DSD, demonstrations that differ mainly in speed share direction targets while retaining distinct scale values. This may make it easier to maintain a consistent motion choice.
We observed the same failure pattern in the Avoiding task from D3IL, a benchmark designed to study diverse demonstrated behaviors. 10 BIN’s trajectories combined incompatible avoidance choices and collided with obstacles. DSD maintained coherent motion around the obstacles. In our 480-rollout evaluation, BIN obtained 0% success and DSD obtained 100%.
| Method | Success rate |
|---|---|
| BIN | 0% |
| DSD | 100% |
Example rollouts from the D3IL Avoiding evaluation.
DSD shows that a simple change in action representation can lead to substantial gains. The mechanisms behind these gains are still being explored, but the results make a compelling case for exploring action representation more closely, both to understand why DSD helps and to discover what other changes might make possible.
References
- Rich Sutton. The Bitter Lesson.
- Google DeepMind. AlphaGo Zero: Starting from scratch.
- Duan et al. Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models.
- DSD project page and demonstrations.
- He et al. Deep Residual Learning for Image Recognition.
- Kim et al. OpenVLA: An Open-Source Vision-Language-Action Model. See also the authors’ clarification of q01/q99 normalization before binning.
- Wu et al. From Foundation to Application: Improving VLA Models in Practice (LingBot-VLA 2.0). Section 6.1, normalization ablation.
- Poke & Wiggle. Independent DSD replication, iteration 2. October 12, 2026; includes normalization and implementation details.
- Yi et al. When Do Discrete Actions Make Good Continuous Controllers? See Section 4 of the paper for factorization error across time and action coordinates.
- Jia et al. Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations. ICLR 2024; D3IL benchmark implementation.