Why SVG Generation Needs a New Scorecard

TL;DR

SVG is widely used on the web, yet producing production-ready vector assets remains costly and difficult to automate. SVG generation spans text, image, web, and multimodal workflows that use either classical tracing algorithms which preserve topology but degrade on complex imagery, or machine learning methods that may be structurally sound but visually inaccurate. Existing evaluation methods are often misaligned with practical usability. Better metrics are needed to capture production requirements such as structural and semantic alignment, design-relevant visual quality, and editability.

What is an SVG file?

Graphic design is a highly iterative process: you sketch, refine, and revise in cycles until the image matches your vision. SVG (Scalable Vector Graphics) is a file format where a drawing is stored as editable instructions such as "draw this curve," "fill this shape," "move or scale this group" instead of pixels.

The animation below is a helpful mental model for a graphic designer's step-by-step SVG production process; SVG can be seen (loosely) as a "mathematical recipe" for the entire drawing process.

Figure 1: Each SVG command adds a small piece, and together they reconstruct the full design like a precise blueprint for the drawing. "Paint Class GIF" — Carlotta Notaro (Behance, published Aug 11, 2019). [16]

SVG code describes shapes, strokes, fills and transforms using a structured syntax. The rendering process (i.e., converting SVG to a viewable image asset such as PNG) can be understood as a drawing pipeline that interprets each element step-by-step to gradually compose the final image.

Figure 2: SVG code generated by GPT-5.2 shown as a step-by-step rendering process.

Why are SVGs Important?

SVG is the most common image format used on the web (about 65% of websites [2]). A major reason is that SVG represents graphics as editable drawing instructions rather than pixels [3]. In practice, this brings several advantages:

1. High-resolution at any size: SVGs scale cleanly without becoming blurry, which is ideal for high-resolution and responsive screens. The same SVG file can be used on a mobile app or a building-sized billboard.

   Figure 3: Raster images (JPG/PNG) are pixel-based and blur when scaled, whereas vector images (SVG) use mathematical paths and stay crisp at any size. [17]

2. Lightweight and easy to maintain: SVG file sizes are often small because the image is stored as compact shape instructions rather than pixel data. And because SVG is text-based, files are easy to edit and reuse across projects [3,4].

   Figure 4: Comparison of Pinterest logo file sizes for a high-quality render across JPG, PNG, and SVG. [18]

3. Searchable and accessible: elements and labels can be indexed and searched, and text can remain real text (supporting accessibility tools such as screen readers) [3,5]. As a result, icons, diagrams, or texts in SVG files are discoverable via site search, docs search, and in-browser 'Find'. This also makes them easier to maintain and edit, since individual elements can be reliably targeted (e.g., by IDs/classes).
4. Web-native and interactive: SVGs integrate well with the web platform. You can style them, script them, and animate them, which makes them widely used for logos, icons, charts, infographics, and illustrations, where clarity and interactivity matter [3,4,5].

   Figure 5: SVG icons are lightweight and stylable, making them ideal for web interfaces.

SVG's advantages are appealing but producing high-quality vector assets remains costly in production.

"Vector-ready" files often require both artistic judgment and practical constraints (clean shapes, consistent structure, and reliable performance). In many teams, that cost shows up as specialized roles and frequent raster-to-vector conversions, and handoff overhead where a single detailed illustration could cost more than $1500 (estimated from [15]).

SVG Generation: State of the Art

Figure 6: SVG generation workflows

In practice, SVG generation can be broadly categorized into four workflows—depending on whether you're creating new assets, converting existing visuals, extracting vectors from existing digital content or combining these approaches:

Table 1: Different SVG generation workflows

Text-to-SVG and image-to-SVG are the core building blocks that multimodal and web-based pipelines often rely on. Hence, this post focuses these two tasks, which share the same goal: turning an input into an editable vector asset.

There are two families of approaches to address the SVG generation problem:

Figure 7: Overview of SVG generation methods

1. Image Processing-Based

Classical image vectorization methods typically uses edge detection methods. For instance, Potrace algorithm traces the boundaries between black and white pixels. These methods have the advantage of effectively preserving topology because they do not process the image semantically, but rather enforce spatial logic. There is no notion of "what object this is," only "where the boundary lies." However, the performance of these methods decreases as the complexity of the image increases.

2. Machine Learning Based, which consists of:

SVG generation guided by image generation models (e.g., diffusion models) has two main steps: 1) generate the initial rasterized image using existing image generation models such as Google's Nano Banana, 2) convert the generated image into SVG code using either classical vectorization algorithms mentioned above or optimization-based methods. While offering high-quality images after rendering, these methods also suffer from a lack of editability in their producsed SVG code.

Model-based code generation, where a trained (vision) language model like OpenAI's ChatGPT outputs SVG code directly token-by-token given a user prompt or image. While offering greater editability, these methods are limited to simple SVG files such as icons, and the quality of the generated image degrades noticeably when testing on samples distinct from the training data.

Evaluation Metrics

With all these approaches for SVG generation, how do we quantitatively decide which SVG is more usable than the others?

The existing metrics in literature can be organized into the following categories:

Table 2: overview of evaluation metrics for SVG generation

Demonstrations

Following is a demonstration of applying these metrics to various SVG generation tasks for five examples. Explore the interactive comparison below to see how different methods perform across Text→SVG, Image→SVG, and Text+Image→SVG tasks.

Figure 8: interactive demo to compare SVG generation methods across different tasks. Use the tabs to switch between task types, and click on examples to see detailed metric comparisons. Use arrow keys (← →) to navigate images and number keys (1-4) to switch tasks.

Rasterized SVG metrics (guitar example): CLIP text-to-image can reward "overall vibe" even when the object is wrong (morphed guitar), suggesting weak keyword or object prioritization. CLIP image-to-image struggles to score shape and color trade-offs (OmniSVG gets the silhouette but misses color). Pixel metrics (e.g., MSE) overemphasize spatial alignment: OmniSVG scores well despite wrong colors, while Claude scores worse despite correct colors but mismatched geometry. Human-preference models can favor simplicity over useful detail (e.g., penalizing added strings).

Code-level metrics: VTracer often yields the highest fidelity image-to-SVG outputs, but its SVG is ~10× longer and dominated by long cubic Bézier paths, making edits painful compared to more structured, commented ML-generated SVG, which highlights a trade-off between fidelity and readability.

<?xml version="1.0" encoding="UTF-8"?>
<!-- Generator: visioncortex VTracer -->

<svg id="svg" version="1.1" xmlns="http://www.w3.org/2000/svg" style="display: block;" viewBox="0 0 512 512"><path d="M0 0 C168.96 0 337.92 0 512 0 C512 168.96 512 337.92 512 512 C343.04 512 174.08 512 0 512 C0 343.04 0 174.08 0 0 Z " transform="translate(0,0)" style="fill: #FEFEFE;"/><path d="M0 0 C2.31 0 4.62 0 7 0 C7.33 9.24 7.66 18.48 8 28 C8.495 27.505 8.495 27.505 9 27 C10.69558726 26.90067847 12.39525747 26.86919993 14.09375 26.8671875 C15.12371094 26.86589844 16.15367187 26.86460937 17.21484375 26.86328125...

In addition to the visual demonstration, we benchmarked the following seven methods: VTracer (a classical image vectorization baseline), OmniSVG, StarVector (Image-to-SVG), NeuralSVG (Text-to-SVG), Gemini, GPT-4o, and Claude across three complexity tiers (Complex, Medium, Simple) for both Image-to-SVG and Text-to-SVG tasks. The results are organized into four metric groups below: code-level scores, pixel & distribution-level scores, alignment scores, and human-preference model scores. Bold values indicate the best result per metric within each task group. Note that CLIP text-to-image is omitted for Image-to-SVG because no text prompt is used in that task.

Table 3: Baseline comparison across complexity levels. Methods are grouped by task type: Image-to-SVG (given a reference image) and Text-to-SVG (given only a text prompt). Bold indicates the best score per column within each task group.

Key takeaway: VTracer, a classical image vectorization method, sweeps most of the fidelity metrics for Image-to-SVG task at the cost of much longer and complex SVG code, making it impractical to edit. Among ML methods, proprietary models (Gemini, GPT-4o, Claude) consistently outperform dedicated open-source models (OmniSVG, StarVector/NeuralSVG) on alignment, preference, and pixel-fidelity metrics while producing compact, reliably renderable SVG.

In summary, while existing metrics are correlated with "quality," they are approximations of what we are aiming for and misleading at times. In addition, we observe a noticeable gap in performance between open-source models and proprietary models, which suggests substantial room for further research in the open-source community.

Figure 9: Misaligned objective

What is a good objective?

Figure 10: Three complementary dimensions of SVG generation for production motivated by our discussion above.

Conclusion

As this post shows, generating production-ready SVG assets is still an open problem. Figure 10 in particular highlights the need for improved evaluation metrics along the three complementary dimensions.

Until we develop metrics that capture these practical dimensions, SVG generation will continue to struggle without a proper goalpost.

Stay tuned for our upcoming technical report, where we’ll introduce our in-house method for machine learning based SVG generation and the improvements it enables.

Footnote: For interested readers, the Awesome-SVG-Generation [12] and Awesome-Sketch-Synthesis [13] lists are good starting hubs.

References:

1. https://en.wikipedia.org/wiki/SVG
2. https://w3techs.com/technologies/details/im-svg
3. https://www.adobe.com/creativecloud/file-types/image/vector/svg-file.html
4. https://www.w3.org/TR/SVG2/
5. https://developer.mozilla.org/en-US/docs/Web/SVG
6. https://www.bls.gov/ooh/arts-and-design/graphic-designers.htm
7. https://www.bls.gov/ooh/arts-and-design/multimedia-artists-and-animators.htm
8. https://helpx.adobe.com/in/illustrator/using/image-trace.html
9. https://github.com/svg/svgo
10. https://lottiefiles.com/supported-features
11. https://github.com/airbnb/lottie/blob/master/supported-features.md
12. https://github.com/Melmaphother/Awesome-SVG-Generation?tab=readme-ov-file
13. https://github.com/MarkMoHR/Awesome-Sketch-Synthesis?tab=readme-ov-file
14. https://github.com/ZJU-REAL/SVGenius/tree/main
15. https://www.upwork.com/hire/illustrators/cost/
16. https://www.behance.net/gallery/84052555/Paint-Class-GIF
17. https://vectorwiz.com/raster-vs-vector-images/
18. https://worldvectorlogo.com/logo/pinterest-2-1
19. https://lightning.ai/docs/torchmetrics/stable/multimodal/clip_score.html
20. https://lightning.ai/docs/torchmetrics/stable/image/frechet_inception_distance.html
21. https://github.com/RE-N-Y/imscore