Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset

Back to Articles From screenshots to HTML code: Introducing the WebSight dataset Published March 15, 2024 Update on GitHub Upvote 13 +7 Hugo Laurençon HugoLaurencon Follow Leo Tronchon Leyo Follow Victor Sanh VictorSanh Follow In the world of web development, turning designs into functional websites usually involves a lot of coding and careful testing. What if we could simplify this process, making it possible to convert web designs into working websites more easily and quickly? WebSight is a new dataset that aims at building AI systems capable of transforming screenshots to HTML code. The challenge Turning a website design or screenshot into HTML code usually needs an experienced developer. But what if this could be more efficient? Motivated by this question, we investigated how vision-language models (VLMs) could be used in web development to create low-code solutions that improve efficiency. Today, the main challenge towards that goal is the lack of high-quality datasets tailored for this task. WebSight aims to fill that gap. WebSight: A large synthetic dataset of screenshot/HTML code pairs In January 2024, we introduced WebSight-v0.1, a synthetic dataset that consists of 823,000 pairs of HTML codes and their corresponding screenshots. This dataset is designed to train AI models to process and translate visual web designs into functional HTML code. By focusing on synthetic data, we've managed to bypass the noise and complexity often found in real-world HTML, allowing AI...