Anyone else looking back at energy-based models for continuous reasoning? [D]

About this article

been re-reading some literature on search and planning lately, and it's getting harder to ignore how brute-forcing next-token prediction is kind of hitting a wall when it comes to strict logic. we keep throwing millions of dollars of compute at scaling transformers, and yeah they get marginally better at standard benchmarks, but the underlying mechanism is still just a massive probability distribution over a discrete vocabulary. when you need absolute mathematical certainty like for formal co...