Diffusion for generating/editing ASTs? [D]

About this article

I’m not a machine learning expert or anything, but I do enjoy learning about how it all works. I’ve noticed that one of the main limitations of LLMs for generating code is that their input and output space is the space of all tokens in the training data. This means that it is entirely possible, and likely, for an LLM to generate code that isn’t even syntactically correct. I’m thinking it would be possible to create some architecture, (diffusion could be a good paradigm) where an abstract synt...