KivEnt SVG Support Coming in 2.2

Thu 14 January 2016

SVG support is coming in the 2.2 release of KivEnt. This post will cover how SVG rendering is handled in Kivy and KivEnt and the basics of turning your SVG files into usable game data in KivEnt.

Everything discussed in this article is currently available in KivEnt's 2.2-dev branch. If you are interested in the example code in this article: it is based on two KivEnt Examples: Example 11 for SVG handling Example 12 for the basic shape construction sections.

Most SVG Rendering is done by a CPU-based algorithm which uses mathematical techniques to calculate the various elements of an SVG drawing. However, if we want to benefit from GPU-hardware acceleration for our rendering, we have to convert this to an approach that matches the way the GPU thinks about graphics: triangles. Everything displayed on screen, from a humble sprite which is 2 triangles forming a rectangle to complex 3d models, they are all made of a group of triangles. There are various ways to triangulate any given shape, and having a solution that accurately triangulates all of the mathematical curves supported by the SVG spec is necessary to support the full format. This GPU based approach will introduce a new factor in calculating our image, we must now rely on the hardware color blending between vertices in our geometry. This produces some noticeable differences in the resulting renders.

GPU Rendering versus CPU Rendering

Kivy added experimental support for the SVG format last year, but we still have had some rendering errors with more complicated SVG files such as the Ghostscript tiger. This is what the tiger looks like with Kivy's current SVG implementation:

Kivy Rendering

This is what the Ghostscript tiger looks like in KivEnt:

KivEnt Rendering

There are still a few rendering errors, the eyes are not quite right, there is an erroneous stroke being rendered over the left side of the right eye. The tongue is also missing the lower dark pink stroke. I hope to solve some of the remaining discrepencies, although I think some of them might be a result of relying on GPU color blending to get the appropriate colorings. Even so, KivEnt's implementation is now 98%-99% accurate, and many SVGs come out looking exactly the same as their CPU rendered equivalents. Aliasing will probably always be slightly different, and in general the whole point of GPU based graphics is to sacrifice a little bit of perfection for the trade-off in performance.

The teeth are a little light compared to Inkscape, but comparable with the coloring in Chrome. The left whiskers are also slightly wider than Inkscape's rendering. Here is the same SVG rendered by Inkscape and Chome:

Inkscape:

InkScape Rendering

Chrome:

Chrome Rendering

I hope to backport the fixes I made for KivEnt's rendering to Kivy once I get the 2.2 release of KivEnt out the door.

If you want to know more about how GPU rendering of SVGs works under the hood, continue reading. The next few sections will provide a basic introduction to rendering colored geometry, the tessellation approach to rendering SVGs, and some discussion of the 'fixes' I applied to improve Kivy's implementation.

The Basics of Triangulation

Before we get into the details of SVG rendering, we need to talk a little bit about how the GPU renders all geometry. The 'basic' unit of geometry for a GPU is the triangle, every 2d or 3d geometry you see on the screen is broken down into triangles before being submitted to the GPU. For instance, a sprite is actually 2 triangles arranged in a rectangle or a circle could be 30 or more triangles arranged like the slices of a pizza or pie.

So for a triangle we would place a vertex at each point on the triangle:

triangle

GL will then blend the color of each of these vertices between each other using some hardware maths. Let's draw this triangle in KivEnt. First of all, all of our rendering is going to be done with the ColorPolyRenderer, this renderer uses a vertex format that looks like:

1
2
3
ctypedef struct VertexFormat2F4UB:
    GLfloat[2] pos
    GLubyte[4] v_color

The name of this format in KivEnt is 'vertex_format_2f4ub'. For those not as familiar with C, this is basically a 2-tuple of floats for the pos, and a 4-tuple of unsigned bytes (0-255) for the v_color. We avoid using 'color' as a attribute name to avoid conflicting with kivy canvas's default 'color' property that is used by many of the widgets.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
def get_triangle_data(side_length):
    return {
        'vertices': {0: {'pos': (-side_length/2., -side_length/2.),
                         'v_color': (255, 0, 0, 255)},
                     1: {'pos': (side_length/2., -side_length/2.),
                         'v_color': (0, 255, 0, 255)},
                     2: {'pos': (0., side_length/2.),
                         'v_color': (0, 0, 255, 255)},
                    },
        'indices': [0, 1, 2],
        'vertex_count': 3,
        'index_count': 3,
        }

Any model in KivEnt is made up of the same basic data as this triangle here:

  1. Vertices is the actual points your model is made up of, in this case our model data is made up of 2 values, the 2d position of that vertex in space, and the rgba color of that vertex. The keys in the vertex dictionary must be the same as the names of the attributes in the vertex format.

  2. Indices is a list of the points in each triangle your model contains. The points are referenced by the number of their vertex.

  3. The vertex count is the number of vertices in your model.

  4. The index count is the number of indices in your model.

Now we just need to load the model's data with our ModelManager and then create an entity that uses this model.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
def draw_triangle(self):
    model_manager = self.gameworld.model_manager
    triangle_data = get_triangle_data(150.)
    triangle_model = model_manager.load_model(
                                        'vertex_format_2f4ub',
                                        triangle_data['vertex_count'],
                                        triangle_data['index_count'],
                                        'triangle',
                                        indices=triangle_data['indices'],
                                        vertices=triangle_data['vertices']
                                        )
    create_dict = {
        'position': (250., 250.),
        'poly_renderer': {'model_key': triangle_model},
        }
    return self.gameworld.init_entity(create_dict, 
                                      ['position', 'poly_renderer'])

We will see:

triangle render

Which you may be familiar with, as this triangle is sort of the 'hello world' of gpu programming. Take note of the way colors blend between the 3 points, this is the approach we will take to turn an SVG into a triangulated mesh. The SVG will basically become a collection of colored triangles, and we will rely on GL's color blending to recreate our SVG.

The most basic 2d rendering unit is often the rectangular sprite. Which is really 2 triangles connected together:

rectangle 1

The triangulation direction is mostly arbitrary, we could also do this one like:

rectangle 2

However, things will render differently, let's render these 2 variations in KivEnt:

Generating the data for these rectangles in KivEnt looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#rectangle 1
def get_rectangle_data(height, width):
    return {
        'vertices': {0: {'pos': (-width/2., -height/2.),
                         'v_color': (255, 0, 0, 255)},
                     1: {'pos': (-width/2., height/2.),
                         'v_color': (0, 255, 0, 255)},
                     2: {'pos': (width/2., height/2.),
                         'v_color': (0, 0, 255, 255)},
                     3: {'pos': (width/2., -height/2.),
                         'v_color': (255, 0, 255, 255)}
                    },
        'indices': [0, 1, 2, 2, 3, 0],
        'vertex_count': 4,
        'index_count': 6,
        }

and for rectangle 2:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#rectangle 2
def get_rectangle_data(height, width):
    return {
        'vertices': {0: {'pos': (-width/2., -height/2.),
                         'v_color': (255, 0, 0, 255)},
                     1: {'pos': (-width/2., height/2.),
                         'v_color': (0, 255, 0, 255)},
                     2: {'pos': (width/2., height/2.),
                         'v_color': (0, 0, 255, 255)},
                     3: {'pos': (width/2., -height/2.),
                         'v_color': (255, 0, 255, 255)}
                    },
        'indices': [0, 1, 3, 3, 1, 2],
        'vertex_count': 4,
        'index_count': 6,
        }

The actual rendering code is no different from the previous triangle example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
def draw_rectangle(self):
    model_manager = self.gameworld.model_manager
    rectangle_data = get_rectangle_data(100., 150.)
    rectangle_model = model_manager.load_model(
                                        'vertex_format_2f4ub',
                                        rectangle_data['vertex_count'],
                                        rectangle_data['index_count'],
                                        'rectangle',
                                        indices=rectangle_data['indices'],
                                        vertices=rectangle_data['vertices']
                                        )
    create_dict = {
        'position': (250., 250.),
        'poly_renderer': {'model_key': rectangle_model},
        }
    return self.gameworld.init_entity(create_dict, 
                                      ['position', 'poly_renderer'])

For rectangle 1 we get:

rectangle 1 rendered

For rectangle 2:

rectangle 2 rendered

More Advanced Shapes

Most shapes require significantly more triangles to represent, for instance it can take at least 32 triangles to create a convincing circle. Rendering a circle is a bit like the slices of a pizza or pie:

circle

However small numbers of edges produce regular polygons, the 8 sides we have drawn in the above diagram would end up looking like:

octagon rendered

If we increase the number of sides to 32, we get something that looks much closer to a circle:

circle rendered

All regular polygons use the same algorithm to draw them, let's take a look at how we generate the data for a regular polygon model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
def get_regular_polygon(sides, r, middle_color, edge_color, pos=(0., 0.)):
    x, y = pos
    angle = 2 * pi / sides
    all_verts = {}
    all_verts[0] = {'pos': pos, 'v_color': middle_color}
    indices = []
    vert_count = 1
    ind_count = 0
    ind_ext = indices.extend 
    c = 1
    for s in range(sides):
        new_pos = [x + (r * sin(s * angle)), y + (r * cos(s * angle))]
        all_verts[vert_count] = {'pos': new_pos, 'v_color': edge_color}
        vert_count += 1
        if c < sides:
            ind_ext((c, 0, c+1))
        else:
            ind_ext((c, 0, 1))
        ind_count += 3
        c += 1
    return {'indices': indices, 'vertices': all_verts, 
            'vertex_count': vert_count, 'index_count': ind_count}

The basic idea is that we add a single point at the center of a circle, and then a number of vertices tracing the circumference equivalent to the number of sides in our polygon. The entity rendering code will be the same as the last 2 examples.

A line would be rendered using a triangulation that looks something like this:

line

The approach is sort of similar to how you would integrate a curve, but gets a bit beyond the scope of hand-writing code to cover every case. Instead, we will make use of a wonderful revamp of a tool that has been around in the GL world for awhile: libtess2.

Tessellating Geometry

The basic approach to rendering an SVG is to take the original points described by the various mathematical elements, feed them into libtess2, take the resulting data and format it appropriately for display in the KivEnt library.

SVG elements are XML data that looks something like this:

1
2
3
4
5
6
7
8
9
<svg id="path6" fill="#FFFFFF" stroke="#000000" stroke-width="0.172" 
d="M58.599,224.09c0,0,0.086,1.619-0.618,1.603
    c-0.704-0.017-14.764-41.095-32.304-39.179C25.677,186.514,40.872,180.231,58.599,224.09z"/>
<path id="path10" fill="#FFFFFF" stroke="#000000" stroke-width="0.172" 
d="M61.616,221.508c0,0-0.471,1.551-1.126,1.296
    c-0.656-0.255,0.099-43.667-17.049-47.833C43.442,174.972,59.867,174.233,61.616,221.508z"/>
<path id="path14" fill="#FFFFFF" stroke="#000000" stroke-width="0.172" 
d="M85.105,257.676c0,0,1.398,0.82,0.997,1.399
    c-0.402,0.578-42.421-10.36-50.5,5.324C35.602,264.399,38.745,248.262,85.105,257.676z"/>

Which is basically a data dump of a variety of information, 'fill' is the color of the interior portion of the curve, 'stroke' the color of the outline, 'stroke-width' the size of the outline, and then the other data carries information about types of curves and the location of the points on them.

From this data a list of points would be generated forming the outline of the shape, then we would add extra vertices into the interior of the shape so that they can hold the information about the color gradient:

tessellating circle

You can then color these vertices to achieve a variety of smooth effects, you can see some examples of this technique in the planets in this video, which are all tessellated circles. In this case, the color of each vertex has been chosen by noise equations in order to generate smooth transitions between the colors.

So for each element of the SVG file, we get a little tessellated piece of geometry with correct colors for the gradients and so on. We draw each element in the order it is declared in the SVG spec and we get a vertex model representing our SVG. This can frequently add up to a very large number of vertices and models. For instance, the tiger that started off this blog post is 239 different models combined together for a total of 117,694 vertices. With the same amount of vertices we could represent 29,423 sprites!

Converting the Tessellation

The output of libtess2 is our geometry in a format known as GL_TRIANGLE_STRIP, the definition of this format is given by the GL programming guide:

1
2
3
4
5
6
7
GL_TRIANGLE_STRIP

Every group of 3 adjacent vertices forms a triangle. The face direction of
the strip is determined by the winding of the first triangle. Each
successive triangle will have its effective face order reversed, so the
system compensates for that by testing it in the opposite way. A vertex
stream of n length will generate n-2 triangles.

Pretty complex sounding, let's illustrate what GL is going on about:

A rectangle would be drawn with vertices:

labeled rectangle

The vertices list for this rectangle would look like A,B,C,D and the index list would be the same, A,B,C,D. GL would then follow the rules for parsing GL_TRIANGLE_STRIP data and create triangles: ABC, CBD

A slightly more complex example, let us revisit the line triangulation:

labeled line

Vertices would be passed in as: A,B,C,D,E,F,G,H,I,J,K,L,M,N

Indices would look the same but be parsed into:

ABC, CBD, CDE, EDF, EFG, GFH, GHI, IHJ, IJK, KJL, KLM, MLN

However, we do not want to use GL_TRIANGLE_STRIP because it adds complications related to the way KivEnt's rendering works. KivEnt renders everything using GL_TRIANGLES which involves explicitly declaring your vertices and the triangles that make them up. This allows KivEnt to automatically add models together so that you don't have to worry about handling efficient submission of your data for each entity, however if we add 2 triangle strip meshes together without modification we will end up with some weird geometry as the implicit triangulation attempts to join the end of one and the beginning of the next. We could add some non-sensical triangles to the end of one model before joining the next, something like the 'triangle': NNN to the end of the line above, accomplished by adding the last vertex an extra 2 times. This would let GL know to not render that particular element, however there are many benefits to working with GL_TRIANGLES, and it is often the most optimized method of rendering given its flexibility.

There was a time when concerns about geometry size resulted in people using solutions such as GL_TRIANGLE_STRIP which better compress their data, but nowadays it is not the size of the data that matters as much as what we are doing with it. Even mobile GPUS are capable of handling millions of vertices so inflating the data related to this a bit when we are still only dealing with 10s of thousands of vertices is no big deal.

So how do we convert from GL_TRIANGLE_STRIP to GL_TRIANGLES?

We do the equivalent of the description above where I turned the vertex list into a list of triangles, remembering to swap every other one (following the GL spec). Python code to do this looks like:

1
2
3
4
5
6
7
8
9
    new_indices = []
    indices = element.indices
    index_count = element.index_count
    for i in range(index_count-2):
        if i % 2 == 0:
            new_indices.extend((indices[i+1], indices[i], indices[i+2]))
        else:
            new_indices.extend((indices[i], indices[i+1], indices[i+2]))
    element.indices = new_indices

Pretty simple conversion, and now we have model data that is ready to be used in KivEnt. The next section will describe how we can take the various elements that come out of a SVG and combine them for efficiency or parse them based on some metadata in order to create individual elements out of a part of the SVG depending on your game logic needs.

Parsing the SVG

With the amount of data contained in many SVG files, it is likely that you will want to process the resulting element models into a more manageable configuration. At the very least, we will typically want to combine models into as large of a grouping as possible. With OpenGL ES2, we use unsigned shorts to hold the index information, which means that ultimately we can provide no more than 65,535 vertices at once or our integer will overflow and we will be rendering crazy triangles. KivEnt's SVG loading has been designed as a 2 step process so that you can customize your final data.

A simple example of the process:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def load_svg(self):
    #First use ModelManager to load the svg file.
    model_manager = self.gameworld.model_manager
    data = model_manager.get_model_info_for_svg("tiger.svg")

    #We are going to need these 2 functions later.
    load_model_from_model_info = model_manager.load_model_from_model_info
    init_entity = self.gameworld.init_entity

    #'model_info' is the SVGModelInfo objects that contain information
    #about a specific element.
    #'svg_name' is needed later, to make sure all elements are loaded
    #under the correct asset.
    model_data = data['model_info']
    svg_name = data['svg_name']

    #This is a cool new feature in 2.2, we are going to copy one
    #entities component.
    entity_to_copy = None

    #This function will combine our elements into as few models as possible
    final_infos = model_manager.combine_model_infos(model_data)
    #Retrieve the bounding box and center information for all vertices.
    svg_bounds = model_manager.get_center_and_bbox_from_infos(final_infos)
    center = svg_bounds['center']
    #We are going to move each model by the negative of the center, so 
    #that the model is centered around 0,0 and adding the entity's position
    #results in the expected location in worldspace.
    neg_center = [-center[0], -center[1]]

    #we need to make an entity for each info in the infos
    #all entities after the first are going to copy the first's
    #position component.
    for model_info in final_infos:
        #retrieve the actual tag for the model asset
        model_name = load_model_from_model_info(model_info, svg_name)
        #Get the model, and shift all vertices by 
        model = model_manager.models[model_name]
        model.add_all_vertex_attribute('pos', neg_center)
        #make the entity.
        create_dict = {
            'position': (300, 300),
            'poly_renderer': {'model_key': model_name},
        }
        #If we are not the first entity, copy that entities position comp.
        if entity_to_copy is not None:
            create_dict['position'] = entity_to_copy
        ent = init_entity(create_dict, ['position', 'poly_renderer'])
        #if we are the first entity, set this entity for copying.
        if entity_to_copy is None:
            entity_to_copy = self.gameworld.entities[ent]

To handle the problem of rendering a model that will require multiple Entity's to hold its data I have introduced a new feature to KivEnt on display here: It is now possible to have one entity copy anothers component, meaning instead of actually creating its own component it just uses the other entity's component. There are a few caveats such as don't destroy the parent entity before the child entity if you do not want unexpected behavior, but for the most part the bookkeeping is already automated. You do this by just providing an Entity object as the create argument in the init_entity call, this will copy just that component of that entity to your new entity. This can help you get around those times when the 1 to 1 nature of entities to components seems to be difficult to get around, such as trying to display multiple models in the same renderer as we are doing here. Now we can just move the first entity created and the others will follow along automatically. Just remember to cleanup all your created entities when the time is appropriate.

In many cases you may just want to load all of the SVG as one entity, but other times you may actually want to parse some of the meta information associated with your SVG elements. KivEnt's SVG loading currently preserves 4 of the SVG metadata tags: 'description', 'title', 'label', and 'id', where it is stored as 'element_id'. You can parse this information as you please when created your lists for the combine_model_info and load_model_From_model_info calls.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
def parse_model_data(self):
model_manager = self.gameworld.model_manager
data = model_manager.get_model_info_for_svg("tiger.svg")
model_data = data['model_info']
for model_info in model_data:
    element_id = model_info.element_id
    label = model_info.label
    description = model_info.description
    title = model_info.description
    #Do your custom grouping here

#load entities as above for each of your model_info groups you created.

The SVG spec is large and we still don't have all of it yet, so please let me know if there is some part of the specification you would use in your game that I am leaving out.

blogroll