tag:blogger.com,1999:blog-84623051020767567332024-03-05T03:45:40.968-08:00OpenGL ES 2.0 games, oolongengine, iPhone, Blender, MeeGo, N9... about OpenGL ES 2.0, cartoonlike animations and games, oolongengine, gamekit, iPhone,... I'm a C++ programmer and would like to make simple 3d games.
I'll write here about progress - hope it helps someone, and hope that someone would correct me or point to different solution. Thanks.astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-8462305102076756733.post-70792259015776874742012-05-13T04:56:00.001-07:002012-05-29T12:42:52.199-07:00Auto rigging, Motion capture import to Blender, Maya->Blender (WAS Blending morphing with skeleton (on GPU) animations. )<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: left;">
After some time, continuing with fixing few remaining issues in GPU skinning for AnimKit. I'd like to prepare another example, with facial animation and normal map skinning. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
AnimKit's AppAnimKitGL blends animations on CPU side - the code <a href="https://github.com/astojilj/astojilj_animkit/commit/3d281eaf071df4f0e90ff46f1daf629b3bdbed99">here</a> does it on iPhone 3GS with morphing animation (head) applied first on CPU, then buffers updated via <span class="s1">glBufferData</span><span class="s2">(... </span>GL_STREAM_DRAW<span class="s2">) and finally skeleton animation applied in vertex shader.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/9Wyn_hPXV7s/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/9Wyn_hPXV7s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="320" height="266" src="http://www.youtube.com/v/9Wyn_hPXV7s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
<div class="separator" style="clear: both; text-align: center;">
AnimKit's AppAnimKitGL animation blending example</div>
<br />
The character is ~2700 vertices it has 18 bones and morphing operates on almost all of the vertices. Though only face vertices get changed, it is a good example to start from. Original "all on CPU" brings ~15 frames per second and "morph on CPU, apply bones on GPU" runs at ~60 fps. For the facial animation I plan to split morphing only to face submesh, then apply bones to all...<br />
<br />
After some time, this become an exercise with different tools, more than handling the animation in C++ code. I saw several questions about feasible way to use artwork from Maya and 3DS MAX in Blender, so I'm going to explain the approach I took. I tried with other formats Collada, obj, but following gave the best results:<br />
<h3 style="text-align: left;">
Converting .mb to .blend and applying textures</h3>
First, started looking for a free 3D model I could use for the example; took this one from <a href="http://fr.3dmodelfree.com/models//26553-0.htm">3dmodelfree.com</a>. It is only available as Maya binary (file name ingame.mb) and had no rig. UV textures are available, but not visible in Maya.<br />
Installed fbx exporter plugin to Maya, opened the file and exported to fbx. Then, imported the .fbx file to 3DS Max and exported to (Extensible 3d) .wrl file. This one, when imported to Blender (using 2.57) showed perfectly, 5 meshes (body, bag, head and 2 eyes) and just needed to get scaled.<br />
For each of the meshes: body, bag, head and 2 eyes, opened Edit mode and used UV unwrap to get the UV texture. Replaced original textures with ones available with .jpeg available with the model. Each of the mesh UV map position was offset and of different size compared to the texture - used Blender UV editor to map unwrapped mesh vertices to texture. Finally, after doing this for all the meshes, model showed properly in Blender:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwfkQGZWF2paPP6LfxEdGEoHP9bXrqI4tgwLljPDAE9RSF5DON9kTe4hRY9z5rFj5f0TkeHp_wA_UCjkzjYwOGaAy-WMyRtdINtpKQQUFJgCkfgzh4zS_QnVCYLfiYOrgncLYGn3VNzX03/s1600/CM+Capture+3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwfkQGZWF2paPP6LfxEdGEoHP9bXrqI4tgwLljPDAE9RSF5DON9kTe4hRY9z5rFj5f0TkeHp_wA_UCjkzjYwOGaAy-WMyRtdINtpKQQUFJgCkfgzh4zS_QnVCYLfiYOrgncLYGn3VNzX03/s400/CM+Capture+3.png" width="303" /></a></div>
<br />
<br />
<h3 style="text-align: left;">
Mixamo auto rigging and using Motion Capture .bvh file</h3>
Google showed me a log of Mixamo advertisements recently and I decided to try it. Exported model as fbx and uploaded it. Notice that the model doesn't have standard T-pose - hands are rotated and close to body. Auto rigger asked me to set position for chin, wrists, elbows and knees and ... In a minute I could see perfectly rigged model in Mixamo web viewer (Unity plugin) with all fingers bones (total ~60 bones) in different positions. Exported the Motion capture file (.bvh) from Mixamo and imported it to Blender model. I could see animated skeleton, same size as original model.<br />
There is a bit of a work to apply skeleton to the mesh, but do it several times and it starts to be fast (about a minute seconds for different motion captures):<br />
Select skeleton, enable X-ray so that it is visible through the mesh, go to Edit mode. Skeleton gets displayed in same pose as the mesh just needs to be translated and rotated to fit the mesh. Once you do it, leave skeleton edit mode (that would cause that skeleton appear somewhere else, but no need to worry). Select all soldier meshes (body, bag, head and 2 eyes) and create a group. This is important to do as skeleton would uniformly deform meshes that are part of the same group - otherwise you would see a bag and eyes moving away from the body.<br />
Having all group members selected in object mode, select also skeleton and use command Object/Parent/Set, and then select armature with automatic weights. You might see that for parts of the original meshes, rig is not properly applied - in that case editing bones enveloper or extruding new bones to cover the are would solve the problem.<br />
<br />
<h3 style="text-align: left;">
In AnimKit on iPhone</h3>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='480' height='360' src='https://www.youtube.com/embed/jcv3mtqI4N0?feature=player_embedded' frameborder='0'></iframe></div>
<div>
<br /></div>
Finally, I saved the file, added it to AppAnimKitGL example and modified code here to use it. 23000 vertices in mesh, ~60 bones and on iPhone 3GS showed in ~45 frames per second. <br />
<br />
Before continuing with morphing and blending, I will need to check feasibility of using .blend file for in game artwork - it is fast for prototyping, but startup time and memory footprint are not that good; should be much better if using PowerVR tooling - for textures only or also for meshes, rigs and animations. Let's see. Found few warning on forums, related to problems with animations when using PowerVR exporters from Blender. Additionally, I spent few learning 3DS Max and character modeling and enjoy some of the features.<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-48175881965860916842012-03-04T05:46:00.002-08:002012-03-04T11:10:31.135-08:00Matrix (palette) skinning for OpenGL ES 2.0 (GLSL, GPU, hardware) - applying bone transformation to mesh in vertex shader<div dir="ltr" style="text-align: left;" trbidi="on">
<title></title>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4e8187}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #78482d}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #31595d}
p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4e8187; min-height: 13.0px}
p.p9 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #d22723; min-height: 13.0px}
p.p10 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #d22723}
p.p11 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008517}
span.s1 {font: 12.0px Helvetica}
span.s2 {color: #4e8187}
span.s3 {color: #743aa7}
span.s4 {color: #411a7f}
span.s5 {color: #78482d}
span.s6 {color: #000000}
span.s7 {color: #3724d4}
span.s8 {color: #bd23a0}
span.s9 {color: #31595d}
span.s10 {font: 12.0px Helvetica; color: #000000}
span.s11 {font: 11.0px Menlo; color: #4e8187}
span.s12 {color: #d22723}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">When starting this post, the intention is to explain how I implemented GPU skinning in previous post, sort of a tutorial on how to port CPU skinning based code (often the approach in desktop code) to OpenGL ES.</span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='480' height='400' src='https://www.youtube.com/embed/x1lnT2td4dM?feature=player_embedded' frameborder='0'></iframe></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p2">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="s1"><b>1) Use buffers for vertices, normals (</b></span><b><span class="s2">m_posnoVertexVboIds)</span><span class="s1"> and vertex indices (</span><span class="s2">m_staticIndexVboIds</span><span class="s1">). </span></b></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="s1">Here, also used for vertex colors (</span><span class="s2">m_staticVertexVboIds</span><span class="s1">). Call this once in init context - e.g. in </span><span class="s3">UIViewController::awakeFromNib implementation - <span class="Apple-style-span" style="color: black;">called when application is brought to foreground.</span></span></span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s4"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;">glBindBuffer</span></span></span><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">(<span class="s5">GL_ARRAY_BUFFER</span>, <span class="s2">m_posnoVertexVboIds</span>[i]);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">glBufferData</span>(<span class="s5">GL_ARRAY_BUFFER</span>, nv*posnodatas, posnodata, <span class="s5">GL_STATIC_DRAW</span>);</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span></span></div>
<div class="p5">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4">glBindBuffer</span><span class="s6">(</span><span class="s5">GL_ARRAY_BUFFER</span><span class="s6">, </span>m_staticVertexVboIds<span class="s6">[i]);</span></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">glBufferData</span>(<span class="s5">GL_ARRAY_BUFFER</span>, nv*staticdatas, staticdata, <span class="s5">GL_STATIC_DRAW</span>);</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div>
<div class="p6">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s4">glBindBuffer</span><span class="s6">(</span>GL_ELEMENT_ARRAY_BUFFER<span class="s6">, </span><span class="s2">m_staticIndexVboIds</span><span class="s6">[i]);</span></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">glBufferData</span>(<span class="s5">GL_ELEMENT_ARRAY_BUFFER</span>, ni*(idatas/<span class="s7">2</span>), indexData2UShort, <span class="s5">GL_STATIC_DRAW</span>);</span></span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s1"><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">No need to read this paragraph further, if you are familiar with stride and </span></span><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="s4">glBufferData. </span><span class="s1">Note that n-th value in </span>indexData2UShort, defines index in posnodata, staticdata related to n-th vertex. e.g. when later "drawing" 56th vertex, if indexData2UShort[55] = 13, means that vertex and normals for 56th vertex are defined in posnodata[13] and color is defined in staticdata[13]. This is quite simplified explanation but 55 is used for 56th element, since counting starts from 0. </span></div>
<div class="p2">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><b>2) Stuff bone index and bone weight data to buffers.</b> </span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Use the same approach as 1) for bone weight and bone indices data. This means that you need to prepare data blob packed like:</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><i>boneIndexData</i>: <span class="Apple-style-span" style="font-size: x-small;"> <vertex1_bone1_index><vertex1_bone2_index><vertex1_bone3_index><vertex1_bone4_index><vertex2_bone1_index><vertex2_bone2_index> ….</span> </span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><i>boneWeightsData:</i> <span class="Apple-style-span" style="font-size: x-small;"><vertex1_bone1_weight><vertex1_bone2_weight><vertex1_bone3_weight><vertex1_bone4_weight><vertex2_bone1_weight><vertex2_bone2_weight> </span></span></div>
<div class="p2">
<br />
<br /></div>
<div class="p7">
<span class="s6"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s8"><span class="Apple-style-span" style="font-size: x-small;">if</span></span><span class="Apple-style-span" style="font-size: x-small;"><span class="s6"> (</span>isMeshDeformedBySkeleton<span class="s6">()) {</span></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s8">void</span> *boneIndexData = sub-><span class="s9">getBoneIndexDataPtr</span>();</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s8">void</span> *boneWeightsData = sub-><span class="s9">getBoneWeightsDataPtr</span>();</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s4">glBindBuffer</span>(<span class="s5">GL_ARRAY_BUFFER_ARB</span>, <span class="s2">m_boneIndexVboIds</span>[i]);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s4">glBufferData</span>(<span class="s5">GL_ARRAY_BUFFER_ARB</span>, nv*<span class="s7">4</span>*<span class="s8">sizeof</span>(<span class="s2">UTuint8</span>), boneIndexData, <span class="s5">GL_STATIC_DRAW</span>);</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s4">glBindBuffer</span>(<span class="s5">GL_ARRAY_BUFFER_ARB</span>, <span class="s2">m_boneWeightVboIds</span>[i]);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s4">glBufferData</span>(<span class="s5">GL_ARRAY_BUFFER_ARB</span>, nv*<span class="s7">4</span>*<span class="s8">sizeof</span>(<span class="s8">float</span>), boneWeightsData, <span class="s5">GL_STATIC_DRAW</span>);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p2">
<br /></div>
<div class="p1">
I<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">n example bellow, <vertexA_boneB_index> is 1 byte (unsigned char), <vertexA_boneB_weight> is 4 byte (float), though 1 or 2 bytes should suffice too.</span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><vertexA_boneB_index> defines which bone index affects position of vertex A, <vertexA_boneB_weight> defines how much it affects. Note that every vertex position can be affected by up to 4 bones. If <vertexA_boneB_weight> == 0, means that bone number <vertexA_boneB_index> doesn't affect vertex A.</span></div>
<div class="p5">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="s10">As explained in 1), index in </span>m_staticIndexVboIds<span class="Apple-style-span" style="color: black;"> maps also to index of data related to vertex in boneIndexData and boneWeightsData arrays. e.g. bytes boneIndexData[13*4],boneIndexData[13*4+1],boneIndexData[13*4+2] and boneIndexData[13*4+3] defines index of bones 1-4 affecting position of 56th vertex (from example in step 1), while 4 byte floats on byte positions boneWeightsData[13*4*4 … 13*4*4+3],…, boneWeightsData[13*4*4+3*4 … 13*4*4+3*4+3] define "weight" of corresponding bones.</span></span></div>
<div class="p8">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="color: black;"><br /></span></span></div>
<div class="p5">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="color: black;">Vertex shader "receives" this data in:</span></span></div>
<div class="p5">
<span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="color: black;"><br /></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;">attribute mediump vec4 a_BoneIndices;</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">attribute mediump vec4 a_BoneWeights;</span></span></div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Data in step 2), like data in step 1), needs to be fed only in init().</span></div>
<div class="p2">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><b>3) update matrix palette with simulation progress </b><b>- prepare matrix palette by stepping simulation </b></span></div>
<div class="p8">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"> </span></div>
<div class="p5">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;"> </span></span><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> m_animengine<span class="s6">-></span><span class="s9">stepTime</span><span class="s6">(time);</span></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">…</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> // fill bone transformation to m_matrixPalette structure</span></span></div>
<div class="p7">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"> pose-></span>fillMatrixPalette<span class="s6">(</span><span class="s2">m_matrixPalette</span><span class="s6">);</span></span></span></div>
<div class="p2">
<span class="Apple-style-span" style="font-size: x-small;"> </span></div>
<div class="p1">
<b><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">4) when rendering every frame, upload updated matrix palette to shader:</span></b></div>
<div class="p4">
<br /></div>
<div class="p5">
<span class="s6"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s4">glUniformMatrix4fv</span><span class="s6">(</span>PiperGL20<span class="s6">::</span><span class="s9">instance</span><span class="s6">()-></span><span class="s9">currentShader</span><span class="s6">()-></span>BoneMatricesHandle<span class="s6">, </span>m_matrixPalette<span class="s6">.</span><span class="s9">size</span><span class="s6">(), </span><span class="s5">GL_FALSE</span><span class="s6">, (</span><span class="s8">float</span><span class="s6">*)matrixData);</span></span></span></div>
<div class="p2">
<br /></div>
<div class="p1">
where:</div>
<div class="p5">
<span class="s6"> <span class="Apple-style-span" style="font-size: x-small;"> </span></span><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">BoneMatricesHandle<span class="s6"> = </span><span class="s4">glGetUniformLocation</span><span class="s6">(</span>uiProgramObject<span class="s6">, </span><span class="s12">"u_BoneMatrices[0]"</span><span class="s6">);</span></span></span></div>
<div class="p4">
<br /></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">and u_BoneMatrices is defined in the vertex shader to support 8 bones (in this example), as:</span></div>
<div class="p4">
<br /></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;">uniform highp mat4 u_BoneMatrices[8];</span></span></div>
<div class="p9">
<br /></div>
<div class="p10">
<b><span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">5) Draw (glDrawElements) - use prepared vertex buffers from step 2) when drawing.</span></span></b></div>
<div class="p9">
<br /></div>
<div class="p4">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></div>
<div class="p6">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-style-span" style="font-size: x-small;"> </span></span><span class="Apple-style-span" style="font-size: x-small;">glBindBuffer<span class="s6">(</span>GL_ARRAY_BUFFER<span class="s6">, </span><span class="s2">m_boneWeightVboIds</span><span class="s6">[j]);</span></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s4">glEnableVertexAttribArray</span>(<span class="s5">GL_BONEWEIGHT_ARRAY</span>);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> glVertexAttribPointer(GL_BONEWEIGHT_ARRAY, <span class="s7">4</span>, GL_FLOAT, GL_FALSE, weightsbuf->stride, (GLvoid*) weightsbuf->getOffset());</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span> <span class="s4">glBindBuffer</span>(GL_ARRAY_BUFFER, <span class="s5">m_boneIndexVboIds</span>[j]);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s5">glEnableVertexAttribArray</span>(GL_BONEINDEX_ARRAY);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s2">glVertexAttribPointer</span>(GL_BONEINDEX_ARRAY, <span class="s7">4</span>, GL_UNSIGNED_BYTE, GL_FALSE, indicesbuf->stride, (GLvoid*) indicesbuf->getOffset());</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s8">if</span> (!useGPUSkinning && PiperGL20::instance()) {</span></span></div>
<div class="p11">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"> </span>// since we use the same shader, turing off matrix skinning this way</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> glUniform1i(PiperGL20::instance()->currentShader()->BoneCountHandle, <span class="s7">0</span>);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p6">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">#endif </span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="s5">glBindBuffer</span>(<span class="s5">GL_ELEMENT_ARRAY_BUFFER_ARB</span>, m_staticIndexVboIds[j]);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> Piper::<span class="s5">instance</span>()->glDrawElements(GL_TRIANGLES, tot, GL_UNSIGNED_SHORT, (GLvoid*)idxbuf->getOffset());</span></span></div>
<div class="p9">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div>
<div class="p9">
<br /></div>
<div class="p10">
<span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">I believe that there is a constraint on number of bones in matrix, but don't know what the number is for iPhone 3GS. If you need more bones then supported by the platform, you could try to split the mesh to sub meshes (and subskeletons) and render submeshes separately.</span></span></div>
<div class="p2">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><b>6) vertex shader does the rest - for every vertex apply bones transformation matrix.</b></span></div>
<div class="p2">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">SkinnedCharacter.vert:</span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: x-small;"> if (u_BoneCount > <span class="s7">0</span>) {</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> highp mat4 mat = u_BoneMatrices[indexOfBone.x] * weightOfBone.x;</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> for (int i = <span class="s7">1</span>; i < u_BoneCount; i++) {</span></span></div>
<div class="p11">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"> </span>// rotate to use indexOfBone.x in the loop</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> indexOfBone = indexOfBone.yzwx;</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> weightOfBone = weightOfBone.yzwx;</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> if (weightOfBone.x > <span class="s7">0.0</span>) {</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> mat = mat + (u_BoneMatrices[indexOfBone.x] * weightOfBone.x);</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span></span></div>
<div class="p4">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> </span></span></div>
<div class="p11">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="s6"> </span>// resulting position after applying skinning</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> vertex = mat * a_Vertex;</span></span></div>
<div class="p3">
<span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> normal = mat * a_Normal;</span></span></div>
<div class="p2">
<br /></div>
<div class="p1">
Complete source code is available <a href="https://github.com/astojilj/astojilj_animkit/commit/fed841528559f0176e1a13829bffe67cbae2665a">here</a>.</div>
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-16713158221337733482012-02-18T03:26:00.001-08:002012-03-03T02:55:43.341-08:00Performances GPU vs CPU matrix palette skinning (WAS: Tutorial #2: Part 1: Animated character walking around. Walk cycle and animated clowds)<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">I planned, for a while, to do another GLES 20 demo for AnimKit with multiple characters on the scene. Found some time yesterday and this is the start (video bellow). Probably I'll cover this in multiple posts; posting the results here when I find some time to progress with it.</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<div class="separator" style="clear: both; text-align: center;">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/cp6hPQYUf0Q?feature=player_embedded' frameborder='0'></iframe></span></div>
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Note that the scene is just a work in progress, clouds behind are big panes causing some performance impact but I think a good illustration of what can be done in few hours and I'll use it as input for later performance optimization work on "AnimKit on OpenGL ES 2.0". I'll post code and more text soon, my one year old is about to wake up.</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">The first work in progress version of code is here: <a href="https://github.com/astojilj/astojilj_animkit/commit/3e7cfe2b2785fcdd367894d03ba61232405d32b1" target="_blank">commit 3e7cfe2b27 WIP: Tutorial #2: Part 1: Animated character walking around. </a></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">The code could be used as illustration and for benchmarking, I guess if you have a simple scene with less then 4000 matrix skinned vertices it should also be usable, but for this example it is not OK - downloaded SDK for iOS 5.0 and tried it on iPhone 3GS. Got only 7 frames per second (~8000 skinned vertices).</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Tried more complex <a href="https://github.com/astojilj/astojilj_animkit/commit/a3998b34cd9253d8462688cc0c5b3dcb8695362b" target="_blank">model with 5 animated instances</a> and got 1-2 frames per second (40000 skinned vertices). The model from post <a href="http://opengles20study.blogspot.com/2011/12/using-animkit-for-pose-and-skeleton.html" target="_blank">Skeleton and pose character animation using AnimKit</a> runs at 14 frames per second.</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">As assumed in previous posts, CPU matrix skinning (<span class="Apple-style-span" style="color: #4e8187;"><a href="https://github.com/astojilj/astojilj_animkit/blob/ios/Source/akGeometryDeformer.cpp#L264" target="_blank">akGeometryDeformer<span class="s1">::LBSkinning()</span></a></span>) that enumerates through all vertices and normals is the bottleneck - most of the time spent there.</span><br />
<title></title>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4e8187}
span.s1 {color: #000000}
</style>
<br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">On the other side, tried PoverWR example of GPU matrix skinning <a href="http://www.imgtec.com/powervr/insider/demos/chameleonman.asp" target="_blank">POWERVR Insider MBX Chameleon Man Demo</a> and it runs quite smoothly,... but it has only ~1000 skinned vertices in the mesh.</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Guess poly-reducing the mesh and moving the calculation to another thread would help, though not significantly (based on the results above) so I plan to do that (reduce complexity of scene) but also check Chameleon Man source code. Code is available as part of PowerVR Insider SDK and you would just need to register to get it.</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><b>Update on March 2nd after implementing GPU skinning</b>: didn't spend time on reducing the scene - just after implemented GPU GLSL matrix skinning the results already look promising (on iPhone 3GS):</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<div style="text-align: center;">
<table align="center" border="1" style="width: 400px;">
<tbody>
<tr>
<th><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Scene</span></th>
<th><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Skinned triangles</span></th>
<th><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">FPS with CPU skinning</span></th>
<th><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">FPS with GPU skinning</span></th>
</tr>
<tr>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">gles2farm - 1 animated character</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">5620</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">8</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">59</span></td>
</tr>
<tr>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">AppAnimKitGL - 5 animated characters</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">33700</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">1</span></td>
<td><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">22</span></td>
</tr>
</tbody></table>
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Code is <a href="https://github.com/astojilj/astojilj_animkit/commit/6af4ae47c45ecee4bbfd9dc1259882e8ac0334d8">here.</a></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;">Five animated characters scene looks like this:</span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<div class="separator" style="clear: both; text-align: center;">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4JT9NvxeY3o6thcn4txFOPnl01H31YOOU4lCjiCbzOaWUhnv8nMW2QITGlpdOcI4HpJMgF0AMYn7l90Rvss7UAKR4KwFnftUuz91nqAInjT6PBhpctLBqjck62qebfMCWzBXaMfFZS7qn/s1600/iPhone_GPU_GLSL_skeleton_animation_5_characters.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4JT9NvxeY3o6thcn4txFOPnl01H31YOOU4lCjiCbzOaWUhnv8nMW2QITGlpdOcI4HpJMgF0AMYn7l90Rvss7UAKR4KwFnftUuz91nqAInjT6PBhpctLBqjck62qebfMCWzBXaMfFZS7qn/s400/iPhone_GPU_GLSL_skeleton_animation_5_characters.png" width="400" /></a></span></div>
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span">In following post, I will try to explain how to implement matrix palette skinning (character skeleton animation using GPU).</span></span><br />
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<title></title>
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Helvetica; min-height: 14.0px}
p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #4e8187}
p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px}
p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo}
p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008517}
p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #78482d}
p.p8 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 16.0px; font: 12.0px Courier; color: #333233}
span.s1 {font: 11.0px Menlo; color: #4e8187}
span.s2 {font: 11.0px Menlo}
span.s3 {color: #000000}
span.s4 {color: #bd23a0}
span.s5 {color: #3724d4}
span.s6 {color: #4e8187}
span.s7 {color: #31595d}
span.s8 {color: #411a7f}
span.s9 {color: #78482d}
span.Apple-tab-span {white-space:pre}
</style>
<br />
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-size: small;"><b>CPU skinning is not the best choice for Opengl ES 2.0 devices.</b></span></span></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-size: small;"><b><br /></b></span></span></span></div>
<div class="p1">
<span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-size: small;">CPU skinning is implemented like this: for every repaint, enumerate through all vertices, calculate and apply bone transformation. In more details, code bellow is an example of CPU skinning in AnimKit's </span></span><span class="s2"><a href="https://github.com/astojilj/astojilj_animkit/blob/ios/Source/akGeometryDeformer.cpp#L286">akGeometryDeformer::LBSkinningUniformScale</a></span>. <span class="Apple-style-span" style="font-size: small;"><span class="Apple-style-span" style="color: black;">Apparently, on iPhone these matrix operations affect performance significantly and are better suited for vertex shader.</span></span></span></div>
<div class="p2">
<br /></div>
<div class="p3">
<span class="s3"><span class="Apple-tab-span"> </span></span><span class="s4">const</span><span class="s3"> </span>btAlignedObjectArray<span class="s3"><</span>akMatrix4<span class="s3">>& matrices = *mpalette;</span></div>
<div class="p4">
<span class="Apple-tab-span"> </span></div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="s4">for</span>(<span class="s4">unsigned</span> <span class="s4">int</span> i=<span class="s5">0</span>; i<vtxCount; i++)</div>
<div class="p5">
<span class="Apple-tab-span"> </span>{</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s6">akMatrix4</span> mat(matrices[indices[<span class="s5">0</span>]] * weights[<span class="s5">0</span>]);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">if</span> (weights[<span class="s5">1</span>]) mat += matrices[indices[<span class="s5">1</span>]] * weights[<span class="s5">1</span>];</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">if</span> (weights[<span class="s5">2</span>]) mat += matrices[indices[<span class="s5">2</span>]] * weights[<span class="s5">2</span>];</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">if</span> (weights[<span class="s5">3</span>]) mat += matrices[indices[<span class="s5">3</span>]] * weights[<span class="s5">3</span>];</div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>// position</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>*vtxDst = (mat * <span class="s6">akVector4</span>(*vtxSrc, <span class="s5">1.f</span>)).<span class="s7">getXYZ</span>();</div>
<div class="p6">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span>// normal</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>*normDst = (mat * <span class="s6">akVector4</span>(*normSrc, <span class="s5">0.0f</span>)).<span class="s7">getXYZ</span>();</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span>*normDst = <span class="s7">normalize</span>(*normDst);</div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(normSrc, normSrcStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(normDst, normDstStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(weights, weightsStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(indices, indicesStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(vtxSrc, vtxSrcStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s7">akAdvancePointer</span>(vtxDst, vtxDstStride);</div>
<div class="p5">
<span class="Apple-tab-span"> </span>}</div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-size: small;">Anyway, when bone transformation is applied and mesh vertices and normals updated, repaint is called after updating vertex buffer <a href="https://github.com/astojilj/astojilj_animkit/blob/master/Samples/akEntity.cpp#L154">updating vertex buffer</a>:</span></span></div>
<div class="p2">
<br /></div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s6">akSubMesh</span>* sub = <span class="s6">m_mesh</span>-><span class="s7">getSubMesh</span>(i);</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s6">UTuint32</span> nv = sub-><span class="s7">getVertexCount</span>();</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s4">void</span> *codata = sub-><span class="s7">getSecondPosNoDataPtr</span>();</div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s6">UTuint32</span> datas = sub-><span class="s7">getPosNoDataStride</span>();</div>
<div class="p4">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></div>
<div class="p7">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s8">glBindBuffer</span><span class="s3">(</span>GL_ARRAY_BUFFER_ARB<span class="s3">, </span><span class="s6">m_posnoVertexVboIds</span><span class="s3">[i]);</span></div>
<div class="p7">
<span class="s3"><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span></span><span class="s8">glBufferData</span><span class="s3">(</span>GL_ARRAY_BUFFER_ARB<span class="s3">, nv*datas, </span><span class="s4">NULL</span><span class="s3">, </span>GL_STREAM_DRAW<span class="s3">);</span></div>
<div class="p5">
<span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="Apple-tab-span"> </span><span class="s8">glBufferSubData</span>(<span class="s9">GL_ARRAY_BUFFER_ARB</span>, <span class="s5">0</span>, nv*datas, codata);</div>
<div class="p4">
<br /></div>
<div class="p1">
<span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="font-size: small;">This part also would get fixed by GPU skinning, since vertices only needs to be "uploaded" once <a href="https://github.com/astojilj/astojilj_animkit/blob/master/Samples/akEntity.cpp#L120">in init()</a> to vertex buffer, instead on every redraw:</span></span></span></div>
<div class="p1">
<span class="Apple-style-span" style="color: black;"><span class="Apple-style-span" style="font-family: Arial, Helvetica, sans-serif;"><span class="Apple-style-span" style="font-size: small;"><br /></span></span></span></div>
<div class="p8">
glBufferData(GL_ARRAY_BUFFER_ARB, nv<b>*</b>posnodatas, posnodata, GL_STATIC_DRAW)</div>
</div>
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com2tag:blogger.com,1999:blog-8462305102076756733.post-13434694236606037752012-01-21T03:19:00.000-08:002012-01-26T15:07:58.661-08:00Tutorial #1: Creating animated characters with Blender and rendering them with AnimKit on iOS<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
So far, I have been writing posts describing how I figured, ported or implemented something. I got a few questions about how to use the code, so I plan to write short tutorials from time to time.<br />
<div>
<br /></div>
<div>
First post is about my findings during creating animated mesh and getting it rendered on iPhone using Opengl ES 2.0 and AppAnimKit. Note that the code should work also on Nokia N9 (Harmattan MeeGo), Ubuntu, Mac OS X.</div>
<div>
<br /></div>
<div>
Here is the result of the steps described in the tutorial: <br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/5NL9J5WfBcg?feature=player_embedded' frameborder='0'></iframe></div>
<br />
Here is what I did:<br />
<br /></div>
</div>
<i>1) downloaded the mesh. </i><br />
I have downloaded this one from <a href="http://www.3dmodelfree.com/models/22496-4.htm#1">http://www.3dmodelfree.com/models/22496-4.htm#1</a>. Note that there are many cute models available there, and also note that they are not for commercial use. I would like to credit the author, but the only thing I could find is that the model is uploaded 3 years ago. If someone knows more about the author of or if it is possible to get author name from 3ds file,… appreciate the help.<br />
The columbine.3ds model contains a mesh, no armature and is not not animated. With ~ 100k vertices it was rather too complex to render on N9 and iPhone. I imported it to Blender from 3ds file.<br />
The model I'm using is using vertex paint to render the model. Rendering using vertex groups is not yet implemented in AnimKit. UV texture works - if your model get's displayed white, you should pull the latest version - plan to push fix later this week.<br />
<br />
<i>2) polyreduce (and triangulate)</i><br />
I find that Blender 2.49b polyreducer script works better then to apply Decimate modifier - which is the approach if using Blender 2.57. So, open mesh in Blender 2.49b, select it, go to Edit mode, select all vertices or only area you want to polyreduce, right mouse button to open menu, from Scripts submenu select Polyreducer. If using per vertex coloring (like I do in this case) disable "Interpolate vertex colors" as result would look fuzzy. I have applied the script several times and video above presents mesh with ~3000 vertices.<br />
<br />
<i>3) Smooth normals</i><br />
While in Blender 2.49b select to Smooth normals. I'm typing this while commuting and don't know out of my head in which button area the toggle is. I could not find this option in Blender 2.57 and, if not set, AnimKit loader would spend noticeable amount of time on startup computing (smoothening) normals.<br />
<br />
<i>4) Create armature</i><br />
Adding armature to mesh start by placing cursor to a dot inside or under the mesh where you want your topmost bone to appear. For this I returned to Blender 2.57. I use term topmost with the meaning "the parent of all other bones". Press space and select Armature. then extrude bones - there are multiple tutorials available about it. To extrude legs and arms use mirror extruding (Shift+E). With Blender 2.57 this option is hidden in Tools toolbar - go to Armature edit mode, open Tools toolbar (Ctrl+T) and check X Axis Mirror Editing. This enables Shift+E - otherwise Shift+E behaves like standard extruding (when pressing E).<br />
<br />
<i>5) Prepare bone influence before parenting armature to mesh</i><br />
Show bones as envelopes and try to scale envelopes (bone effect area) to cover parts of mesh you wish them to affect. With current version of Animkit it is good not to leave mesh areas that are not under bone influence. I plan to open an issue/push patch about this once I get time to debug it. If later you see parts of the mesh stretched to the mesh centre, it is a sign you need to come back to this step.<br />
<br />
<i>6) parent the mesh to the armature. Select automatic weights. </i><br />
<br />
<i>7) Start creating animation (create action)</i><br />
Go to pose mode. Open one DopeSheet view and switch to Action Editor. Add an action. Make sure timeline cursor is on frame 1.<br />
<br />
<i>8) Add key </i><br />
rotate and translate tail bone to extreme position on the left. Press I to insert key.<br />
<i><br /></i><br />
<i>9) Add one more key</i><br />
move the time slider and position the tail as in initial position. Press I to add another key.<br />
<i><br /></i><br />
<i>10) Add one more key</i><br />
add symmetrical position to position in step 8). It is an extreme position with tail on right.<br />
<i><br /></i><br />
<i>11) Finish animation editing (more keys)</i><br />
copy 9) and 8) keys to get full cycle of tail going 8(left) -> 9(down) -> 10(right) -> (down) -> (left).
After this I opened timeline view, set animation length to length of the cycle in 11) and could see animation rendered in Blender.<br />
<br />
<i>12) use file from C++ code.</i><br />
I saved the file and changed the demo code from Blu.blend to use my mesh. Don't forget to add the file to build target (has to be in the bundle).<br />
<br />
<i>13) Contributing patch upstream (gamekit)</i><br />
There were few minor issues as initially I could not see animation running. It was a nice experience to start collaboration with Ervin and Xavier from gamekit project - getting a minor fix for animation not playing in AnimKit upstream code: just opened an issue and attached the patch. Fix got in AnimKit soon after that. Few other related patches I plan to "pull" to https://github.com/astojilj/astojilj_animkit/tree/ios later this week, but the code there should already be OK.</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-21070783397076892832011-12-15T13:49:00.001-08:002012-01-10T11:20:47.961-08:00Skeleton and pose character animation using AnimKit<div dir="ltr" style="text-align: left;" trbidi="on"><i>Objective:</i> How to implement pose and skeleton character animation in OpenGL ES 2.0? <br />
<br />
I don't want to use gamekit with OGRE 1.8 rendering engine - need something lighter. Don't see this as "reinventing the wheel" - though there is nothing wrong with "reinventing the wheel" when it is fun, when you enjoy doing it. Hunter S. Thomson typed copies of Hemingway's books - he enjoyed doing it and he knew why he was doing it. I believe he wasn't having a feeling he would live for 1000 years - so, why not waste a bit of time... IMHO, doing this is the best/fastest way for me to learn and it is fun. Guess one could say that I could also type copy of Hunter S. Thomson articles, before continuing, to make this blog more fun to read.<br />
<br />
Anyway, I think this is just what I need - <a href="http://code.google.com/p/gamekit/wiki/AnimKit">gamekit's AnimKit</a>. Briefly checked the code and run it on my desktop: it supports vertex (pose and morph) and skeleton animations, animation blending... inverse kinematics in the roadmap. Grepped for MATRIX_PALETTE - seems to have software (on CPU) animation implemented only. There is nice demo application AppAnimKitGL which looks quite promising. It is dependent on libGLU, libGLEW and libGLUT, and with fixed rendering pipeline. So, there is some effort needed to port it to iPhone and N9 - but I did this work in previous posts/examples so it should go faster now. Let's see. Plan to first focus on opengles 2.0.<br />
...<br />
Continuing:<br />
Had some time during the weekend to check this. Here is how it looks on Nokia N9 running next to the AnimKit demo on Ubuntu:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/VNOpSIlSsYg?feature=player_embedded' frameborder='0'></iframe></div><div class="separator" style="clear: both; text-align: center;"><br />
</div>Note that I did not port all of the demo helper features (bones, normal rendering, etc). Difference in texture color on N9 comes from using loaded BGR texture as RGB - if you plan to use the same code, you'll probably use different Blender model and different texture loader.<br />
Code is available here: <a href="https://github.com/astojilj/astojilj_animkit/commit/df571339ab55d74681badf8c88cc468cedb3d372">https://github.com/astojilj/astojilj_animkit/commit/df571339ab55d74681badf8c88cc468cedb3d372</a><br />
To run it on Nokia N9, open <span class="Apple-style-span" style="background-color: #eeeeee; color: #333333; font-family: Monaco, 'Courier New', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; font-size: 12px; line-height: 33px;">Samples/AnimKitGL/AppAnimKitGl.pro </span>with QtCreator and just build&deploy. If you intend to run original demo on desktop, note that you'll need to have Blu.blend in current directory.<br />
<br />
Next thing I plan to do is add (copy from previous project) iPhone/iPad project support files.<br />
<br />
...<br />
<br />
<i>iPhone version</i><br />
<i><br />
</i><br />
<div class="separator" style="clear: both; text-align: center;"><object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/9Wyn_hPXV7s/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/9Wyn_hPXV7s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" /> <param name="bgcolor" value="#FFFFFF" /> <embed width="480" height="400" src="http://www.youtube.com/v/9Wyn_hPXV7s?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div><i><br />
</i><br />
Note that the fragment shader is different from N9 version - uncommented the code that is darkening the borders and doing the phong (giving the cartoon shading look).<br />
<br />
Started from XCode, File->New Project->OpenGL ES Application (iPhone). After this, added all the sources to the project, replaced all "in project" project includes using <> with "" (e.g. #include "Mathematics.h") and added OPENGL_ES_2_0 to <i>Preprocessor Macros</i> item in Target Info dialog. Added also code to printout frames per second info.<br />
<br />
This made the things compile, but when run I couldn't see anything rendered. Both use the same memory layout (little endian) so that wasn't a problem. Turned out that with 15000+ indices using glDrawElements with GL_UNSIGNED_INT indices doesn't work. GL_UNSIGNED_SHORT does.<br />
Put a quick fix in the code with following comment:<br />
<title></title> <style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #008517}
</style> <br />
<div class="p1"><br />
</div><div class="p1">// in iOS 4.1 simulator, glDrawElements(GL_TRIANGLES, cnt, GL_UNSIGNED_INT did not work - did not show anything on screen</div><div class="p1">// while GL_UNSIGNED_SHORT works. Using this as temporary workaround until trying with new sdk or anyway, change types</div><div class="p1">// in animkit code (no reason to use unsigned int anyway).</div><br />
<br />
This got the scene rendered, and only thing left to do was to <a href="https://github.com/astojilj/astojilj_animkit/blob/ios/Samples/AnimKitGL/AppAnimKitGl/Classes/EAGLView.m#L91" target="_blank">add depth buffer support</a> (as default XCode SDK skeleton code doesn't have it on).<br />
<br />
Code for the example running on iPhone (note that I was using iOS SDK 4.1 - yep, plan to update soon...) https://github.com/astojilj/astojilj_animkit/commit/5c11f660281bb8c8a6b5e3bba93938f703b6307c. Few fixes added later here: <a href="https://github.com/astojilj/astojilj_animkit/commit/03545c7d1ea753c078aab4d441d4f12d91d462b8">https://github.com/astojilj/astojilj_animkit/commit/03545c7d1ea753c078aab4d441d4f12d91d462b8</a><br />
Plan to merge it to master after verifying changes don't break anything on N9.<br />
<br />
Next thing to try is scene with multiple actors with animated skeleton (I think I'll do cartoon animals) hardware (on GPU) matrix palette skeleton animation.</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com2tag:blogger.com,1999:blog-8462305102076756733.post-76869972149291945872011-12-06T04:09:00.000-08:002011-12-06T04:11:09.933-08:00Collision detection for character - hit character falls down as rag-doll<div dir="ltr" style="text-align: left;" trbidi="on"><br />
<i>Objective:</i> how to implement collision detection for animated character? Character has bone system and it's position and movement is controlled by pose and animations but once it gets hit it needs to fall down like a rag-doll.<br />
<br />
<i>Get</i> <i>character model to ReadBlend:</i> There is no code change needed for this part - if you have existing model with character, open project's blend file (PhysicsAnimationBakingDemo.blend) and use Blender's File->Append or Link command. Once you select blender file to "append" character model, select Objects and then selects meshes, armatures or constraints to import. I tried with <a href="http://blenderartists.org/forum/showthread.php?131079-Simple-biped-rig-free-to-use" target="_blank">this file</a> and it worked - got the static character displayed in the demo. However, with ~300 triangles per each piece it was needed to reduce number of vertices not to let it affect frames per second count. If you hit an error "Error Not A Library" in Blender when appending, usually it is related to incompatible versions - open biped-rig blend file, save it (overwrite) and then try append from it again. <br />
<br />
Once I got biped_rig's character displayed next to the tree, it just didn't look nice. Decided to do my own made from boxes; it didn't take long to make head, spine, pelvis, upper and lower legs and arms, feet (and hands) from boxes used in the scene. When model is created, assuming all body pieces are marked as default "Static" (and not as "Rigid body") in Blender, character made of boxes gets displayed in the demo in standing position, not affected by gravity but with ball bouncing against it. If body parts would be marked as "Dynamic", ball would push them. So, for the character body parts, controlled by animation proper mode is Dynamic. Once the contact with bullet is detected, we will just "switch" them to be affected by gravity and they'll fall down.<br />
<br />
<i>Modeling joint constraints in Blender:</i> to get "more realistic" rag-doll joint constraints, I need btConeTwistConstraint and btHingeConstraint and to set limits. Check the details for joint constraints example from <a href="http://code.google.com/p/bullet/source/browse/trunk/Demos/RagdollDemo/RagdollDemo.cpp" target="_blank">Bulletphysics Ragdoll example code</a>. I didn't set the limits in Blender 2.49, which you can tell from the demo bellow. When needed, constraints could be tweaked in C++ code (BulletBlendReaderNew::convertConstraints()). Anyway, just put joint constraints between body parts (<a href="http://www.blender3dclub.com/pdfs/bridge.pdf" target="_blank">this tutorial</a> might help) to Blender file and they work also on phone.<br />
<br />
<i>Turn on the gravity for character body parts - when character is hit, it needs to fall down like a rag-doll:</i> just after stepSimulation(), call processContactData() to check collision between ball and character:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">void BulletBlendReaderNew::processContactData()</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">{</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> //Assume world->stepSimulation or world->performDiscreteCollisionDetection has been called</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> int numManifolds = m_destinationWorld->getDispatcher()->getNumManifolds();</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> for (int i=0;i<numManifolds;i++)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> btPersistentManifold* contactManifold = m_destinationWorld->getDispatcher()->getManifoldByIndexInternal(i);</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> btCollisionObject* obA = static_cast<btCollisionObject*>(contactManifold->getBody0());</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> btCollisionObject* obB = static_cast<btCollisionObject*>(contactManifold->getBody1());</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> int numContacts = contactManifold->getNumContacts();</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> if (!numContacts)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> continue;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>// check if one of colliding shapes is ball</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> if (obA != ball && obB != ball)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> continue;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>// this is simplified check if another collision shape is part of character's armature</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> if (obA->getCollisionShape()->getUserPointer() == &(this->armature)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> || obB->getCollisionShape()->getUserPointer() == &(this->armature))</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> // convert armature -> ragdoll in case ball hits character</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> turnToRagdoll(&(this->armature));</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">}</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">void BulletBlendReaderNew::turnToRagdoll(ArmNode *nodeList)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">{</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> ArmNode *next = nodeList;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> while(next) {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> if (next->piece)</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> ((btRigidBody*)next->piece)->setAngularFactor(1.f);</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> next = next->nextSibling;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">}</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/R7FS8JxgS8A?feature=player_embedded' frameborder='0'></iframe></div><br />
<br />
Note that all the body parts' meshes are marked as "Dynamic" in Blender's Logic panel (F4). Dynamic bodies have setAngularFactor set in ReadBlend set to 0.f initially in BulletBlendReaderNew::createBulletObject().<br />
<br />
In case of complex sophisticated character meshes, it makes sense to use the same technique, and not only on constrained systems (like iPhone and N9) - don't add complete mesh to physics world for collision and gravity simulation, but create "shadow" model out of boxes only for physics and collision. Then, when rendering rigged mesh, use motionState to get location (transformation) of each box (bone in simplified armature in physics simulation world) and then apply transformation to complex rigged character's bones when rendering it. Simple physics model would not be rendered, it would just exist in physics world.<br />
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-17869447983968401082011-11-28T08:26:00.000-08:002011-11-28T09:06:07.355-08:00Tree collision demo - getting the texture and color to N9<div dir="ltr" style="text-align: left;" trbidi="on">I didn't use textures from demo <a href="http://opengles20study.blogspot.com/2011/01/cel-shading-toon-shading-silhouette-how.html">http://opengles20study.blogspot.com/2011/01/cel-shading-toon-shading-silhouette-how.html</a> after porting to N9: <a href="http://opengles20study.blogspot.com/2011/08/meego.html">http://opengles20study.blogspot.com/2011/08/meego.html</a><br />
To get the textures work if using jpeg library (if not using iPhone SDK to generate images - code under <span class="Apple-style-span" style="background-color: white; font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace; font-size: 12px; white-space: pre;">#ifdef USE_IPHONE_SDK_JPEGLIB</span>) it is needed to change <span class="Apple-style-span" style="color: maroon; font-family: monospace; white-space: pre;">m_pixelColorComponents</span> from <span class="Apple-style-span" style="background-color: white; font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace; font-size: 12px; white-space: pre;">GL_RGBA to </span><span class="Apple-style-span" style="background-color: white; font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace; font-size: 12px; white-space: pre;">GL_RGB </span><a href="http://code.google.com/p/oolongengine/source/browse/trunk/Oolong%20Engine2/Examples/Demos/ReadBlend/OolongReadBlend.mm#110" target="_blank">here</a>.<br />
It seemed like a good idea not to texture the tree, but to supply color via uniform value when rendering the tree per instance. I plan to check how the tree looks with very few leaves and for that ngPlant leaves group probably would be better to use textures (billboards). For example about how to use uniform color value in shader, just grep the code for u_Color - there are several available.<br />
This is how it looks on N9 - sorry for the poor video quality.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/dOuvr3Z7P18?feature=player_embedded' frameborder='0'></iframe></div><br />
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-90018820069201438012011-11-07T06:26:00.001-08:002011-11-09T10:00:24.638-08:00Tree branches collision detection<div dir="ltr" style="text-align: left;" trbidi="on"><div dir="ltr" style="text-align: left;" trbidi="on"><div dir="ltr" style="text-align: left;" trbidi="on"><div dir="ltr" style="text-align: left;" trbidi="on"><div dir="ltr" style="text-align: left;" trbidi="on">I showed the example from previous post to cousin Miša and explained how I plan to add physics and collision detection for each branch and leaf of the tree; he noticed that the tree looks quite complex (several thousands vertices) and asked how much CPU that could consume, and how much it could affect performance and lower frames per second count...<br />
In short, this is what the post, code examples and demo is about. There is another thing that could be interesting, animating camera position from behind the ball to the position orthogonal to ball direction while the ball is moving - not stopping or slowing down the simulation like in previous posts.<br />
<br />
Have to apologize for the not so good quality of videos I made; I'll try to compensate it with detailed explanation and code examples about how to use related ngPlant and bulletphysics API.<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/6yz2aI2Q1Fs?feature=player_embedded' frameborder='0'></iframe></div><br />
The video above shows a heavy ball rolling down the hill to hit the tree, bounce back and hit low hanging tree branch.<br />
Another video shows a rather heavy ball rolling down the hill, hitting the tree,... after it rolls away from the tree btRigidBody::applyCentralImpulse() is "applied" throwing the ball back to the tree. After it bounces back several times and get thrown again, eventually ball gets stuck between the branches of the tree.<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/YwsgasReh3I?feature=player_embedded' frameborder='0'></iframe></div><br />
This is simplified version of the code throwing the ball from the ground to the tree - impulse with angle of 45 degrees (500 horizontally and 500 vertically):<br />
<code><br />
<span style="font-size: x-small;">Vec3 vToTree;<br />
MatrixVec3Normalize(vToTree, vTarget - vFrom); // vTarget is tree, <br />
// vFrom is current ball position<br />
((btRigidBody*)ball)->applyCentralImpulse(btVector3(0.,0.,500.) + 500. * btVector3(vToTree.x, vToTree.y, vToTree.z));<br />
</span></code><br />
<br />
<b>Camera animation while simulation is running - from behind the ball to orthogonal to ball rolling direction</b></div><br />
For this, I'm using TransformInterpolator class designed in <a href="http://opengles20study.blogspot.com/2011/01/camera-movement-animation-slerp-and.html">previous post about camera movement animation</a> - <i>start point</i> in transform interpolation is defined as camera behind the ball:<br />
<b> </b><br />
<code><br />
<span style="font-size: x-small;"> MATRIX mTransformStart;<br />
// take vFrom as previous position and vTarget as current, or in this case it is tree position<br />
modelTransformWithCameraBehindTheBall(mTransformStart, vFrom, vTarget);<br />
cameraAnimation = new TransformInterpolator(mTransformStart, mTransform, 75);<br />
</span></code><br />
<br />
<i>End point</i> in camera animation (mTransform) is camera transformation to look at ball orthogonal to direction of ball movement, calculated as in this example:</div><code><span style="font-size: x-small;"><br />
Vec3 vBallDirection = vTarget - vBall;<br />
Vec3 vViewDirection, vDirection;<br />
// camera position on vector vViewDirection from the ball:<br />
// it needs to be orthogonal to both ball movement direction (vBallDirection) and "up" vector z<br />
// as in example coordinate system z means up.<br />
MatrixVec3CrossProduct(vViewDirection, vBallDirection, Vec3(0,0,1));<br />
<br />
MatrixVec3Normalize(vDirection, vViewDirection);<br />
// center of the view would be between ball and the tree<br />
Vec3 vTo = vBall + vBallDirection/ 2;<br />
<br />
// we take some distance from "center of the view" - a point that lies on vector vViewDirection<br />
// looking at vTo is then nice landscape view, orthogonal to "up" and ball direction. <br />
MatrixLookAtRH(mTransform, vTo + 3.f * CameraDistanceFromBall * vDirection, vTo, Vec3(0,0,1));<br />
</span><br />
</code><br />
It is important to update cameraAnimation endPoing after each dynamicsWorld->stepSimulation(); as the ball advances to the tree camera needs to follow - after animation is finished, camera will just follow the ball movement. Since the target is the tree, this would give a nice side view of ball hitting and bouncing of the tree, as on both videos above. <br />
<br />
<b>Physics (collision) for tree and all tree branches</b><br />
<br />
ngPlant and bulletphysics btIndexedMesh and btTriangleMesh APIs fit together; I'm using:</div><code><br />
void P3DHLIPlantInstance::FillVAttrBuffersI( const P3DHLIVAttrBuffers *VAttrBuffers, unsigned int GroupIndex) const<br />
</code><br />
to load vertices, used to render the tree and for physics btIndexedMesh::m_vertexBase.<br />
<br />
<code><br />
void P3DHLIPlantTemplate::FillIndexBuffer<br />
(void *IndexBuffer,<br />
unsigned int GroupIndex,<br />
unsigned int PrimitiveType,<br />
unsigned int ElementType,<br />
unsigned int IndexBase) const<br />
</code><br />
to load index of vertices in triangles, used to render the tree and for physics btIndexedMesh::m_triangleIndexBase</div><br />
This is a code used to add mesh for a tree to btDynamicsWorld.<br />
<br />
<code><br />
<span style="font-size: x-small;">btVector3 scale(object->size.x,object->size.y,object->size.z);<br />
btTransform worldTrans;<br />
worldTrans.setIdentity();<br />
worldTrans.setOrigin(btVector3(object->loc.x,object->loc.y,object->loc.z));<br />
worldTrans.getBasis().setEulerZYX(object->rot.x,object->rot.y,object->rot.z);<br />
<br />
<br />
btTriangleMesh* meshInterface = new btTriangleMesh();<br />
// for now, modeling this way trees in blender - a mash with name OBplant_ <br />
if (!strncmp(object->id.name, "OBplant_", 6)) {<br />
<br />
// get the plant data<br />
PlantGraphicsObject *plant = (PlantGraphicsObject *)createGraphicsObject(object, 0);<br />
<br />
// fill the mesh bullet interface with plant branch layer data<br />
for (int i = 0; i < plant->plantData().branchLevels.size(); i++) { </span></code><br />
<span style="font-size: x-small;"><code> btIndexedMesh mesh; </code></span><br />
<code><span style="font-size: x-small;"> const PlantObjectData::BranchLevel &levelData(plant->plantData().branchLevels.at(i));<br />
mesh.m_numTriangles = levelData.IndexCount / 3;<br />
mesh.m_triangleIndexBase = (const unsigned char *)levelData.IndexBuffer;<br />
mesh.m_triangleIndexStride = 3 * sizeof(unsigned int);<br />
mesh.m_numVertices = levelData.vertexCount;<br />
mesh.m_vertexBase = (const unsigned char *)levelData.PosBuffer;<br />
mesh.m_vertexStride = sizeof(float) * 3;<br />
meshInterface->addIndexedMesh(mesh);<br />
}<br />
<br />
// create convex collision object based on mesh bullet interface<br />
btCollisionShape* colShape = 0;<br />
btBvhTriangleMeshShape* childShape = new btBvhTriangleMeshShape(meshInterface, true);<br />
<br />
if (scale[0]!=1. || scale[1]!=1. || scale[2]!=1.) {<br />
// I have scaled down the mesh in blender as it was too big<br />
colShape = new btScaledBvhTriangleMeshShape(childShape,scale);<br />
} else { </span></code><br />
<code><span style="font-size: x-small;"> colShape = childShape;<br />
}<br />
<br />
btVector3 inertia(0,0,0);<br />
btRigidBody* colObj = new btRigidBody(0.f,0,colShape,inertia);<br />
colObj->setWorldTransform(worldTrans);<br />
<br />
// references ...<br />
colObj->setCollisionShape(colShape);<br />
plant->setCollisionObject(colObj);<br />
colObj->setUserPointer(plant);<br />
<br />
// add the physics object to the world<br />
m_destinationWorld->addRigidBody(colObj);</span><br />
</code></div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-62119155937036127932011-10-16T15:03:00.000-07:002011-11-07T04:45:44.698-08:00N9<div dir="ltr" style="text-align: left;" trbidi="on">I'm very proud I was part of <a href="http://swipe.nokia.com/">N9</a> team for last three years.<a href="http://swipe.nokia.com/"> The device</a> is out. It is beautiful piece of hardware with amazing screen.<br />
<br />
I'll start using it as development device for this thing I'm learning here. Here is the demo <a href="http://opengles20study.blogspot.com/2011/09/modeling-plants-trees-forest-ngplant.html">from the last post</a>, with 2 ngPlant trees, running on N9.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='480' height='399' src='https://www.youtube.com/embed/j8bvOQHWAcE?feature=player_embedded' frameborder='0'></iframe></div><br />
Next thing I'm planning to do is to add collision for all of the branches in ngPlant tree to PlayBlends' btDiscreteDynamicsWorld - let's say simulating a ball hitting a branch, then bouncing from one to another until it fells down to ground ... or micro airplane flying through treetop ... anyway, just would like to see how it performs and what are the constraints. Rendering ngPlant tree seems quite OK - uploading vertex and index data to GPU buffers for rendering:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"> glBindBuffer(GL_ARRAY_BUFFER,...);</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> glBufferData(GL_ARRAY_BUFFER, ... GL_STATIC_DRAW);</span></span><br />
<div style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">....</span></div><div style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"> Piper::instance()->glDrawElements(GL_TRIANGLES....</span></div><div style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><br />
</span></div>Note that ngPlant doesn't yet support exporting mash data as <span class="term">triangle strips. <a href="http://ngplant.sourceforge.net/docapi/hliapicpp.html">The documentation</a> states it is planned for</span> next versions.<br />
For tree branches collision detection, I'll start by reading <a href="http://www.amazon.com/Real-Time-Collision-Detection-Interactive-Technology/dp/1558607323">Real Time Collision Detection</a> and check <a href="http://bulletphysics.com/Bullet/BulletFull/classbtBvhTriangleMeshShape.html">btBvhTriangleMeshShape.</a><br />
<br />
<a href="http://www.blogger.com/swipe.nokia.com"></a></div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-5890916899456802712011-09-25T14:48:00.000-07:002011-12-07T10:00:05.694-08:00Rendering plants, trees, forest. ngPlant.<div dir="ltr" style="text-align: left;" trbidi="on">ngPlant (http://ngplant.sourceforge.net/) is an open source plant modeling software and I wanted to check how to use it and how it performs. Downloaded ngPlant source from http://sourceforge.net/projects/ngplant/files/ngplant/0.9.8/ngplant-0.9.8.tar.gz/download. <br />
<br />
ngPlant<a href="http://ngplant.sourceforge.net/#licenseanc"> license</a>:<br />
"<br />
ngPlant is a Free Software project. ngPlant modeling tool and ngpshot utility are distributed under the terms of the GNU General Public License (GPL). Software libraries (libngpcore, libngput and pywrapper) which may be used in another projects are distributed under the terms of the BSD License. <br />
"<br />
Checked the code; to generate models (ngPlant documentation uses the term "instantiate") and render a group of (different) trees of the same species, I would need libngpcore and maybe libngput. "Maybe libngput" as some of the code seems already existing in oolongengine code.<br />
BSD license (libngpcore and libngput) means that code is OK to statically link to and use in iPhone apps. GPL license for ngpview and ngpshow prohibits reusing their rendering implementation in iPhone or N9 (non GPL) applications. Anyway, checked the code and since it is fixed pipeline desktop OpenGL code using GLX and GLEW, I could not use it as is anyway.<br />
In short, I could reuse loading from ngp files and instantiating tree mash models but needed to write rendering.<br />
<br />
Compiling went OK with no bigger problems - there is dependency to GLX in p3dglmemcntx.cpp (and .h). Removed them one and png backend from compilation. Next was to write OpenGL ES 2.0 renderring for ngPlant and include it's physics. First, I took example elm.ngp available on Yorik's <a href="http://yorik.uncreated.net/tutorials/treemaking.html">plant making tutorial</a>. There is also a plant library available <a href="http://yorik.uncreated.net/greenhouse.html">here</a>. Thanks Yorik.<br />
<br />
Anyway,... elm.ngp. With tens of thousands of triangles, the tree model was a bit heavy to render - so decided to simplify it a bit: removed few branches and did not even want to display leaves in this scene (term <i>billboards</i> used in ngPlant). Elm tree was too big comparing to the rest of the scene, so scaled it down when rendering. I've positioned the tree so that the ball would, after hitting boxes, bounce off it. Here is the result:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/JmRrDQ7upmM?feature=player_embedded' frameborder='0'></iframe></div><br />
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-88397798038255228252011-08-06T22:05:00.000-07:002011-08-06T22:06:43.020-07:00Porting to Linux (MeeGo)<div dir="ltr" style="text-align: left;" trbidi="on"><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/TE7YIukdalA?feature=player_embedded' frameborder='0'></iframe></div>Video corresponds to the first version of Linux (EGL, XLib, OpenGL ES 2.0) port. N900 is used for the demo and I plan to provide comparable video on N9 later, after sales start.<br />
<br />
Pushed the source code to branch <a href="https://github.com/astoj/astoj_oolongengine/tree/e20584fa0ea63ffe9576729c04fb5ea3857eb8eb">portToMeego</a>. It uses EGL and XLib to create window and initialize GL surface. Used Qt project files, and run the code from Qt Creator. <a href="http://www.developer.nokia.com/info/sw.nokia.com/id/da8df288-e615-443d-be5c-00c8a72435f8/Qt_SDK.html">Qt SDK 1.1.2</a> includes support for N9 (MeeGo Harmattan), MeeGo and N900.<br />
<br />
There is a nice example <a href="http://wiki.maemo.org/SimpleGL_example">on wiki.maemo.org</a><span id="goog_907517683"></span><span id="goog_907517684"></span> demonstrating how to use EGL, XLib and OpenGL ES 2.0. To run it on MeeGo devices, you'll just need to remove N900 specific call to XChangeProperty with "_HILDON_NON_COMPOSITED_WINDOW".<br />
Comparing to iOS's EAGLContext::presentRenderBuffer (example behind the <a href="https://github.com/astoj/astoj_oolongengine/blob/e20584fa0ea63ffe9576729c04fb5ea3857eb8eb/Renderer/Core/GraphicsDevice/GLES20/EAGLView2.m#L342">link</a> includes also initialization), with EGL swap renderbuffer using eglSwapBuffers, like <a href="https://github.com/astoj/astoj_oolongengine/blob/e20584fa0ea63ffe9576729c04fb5ea3857eb8eb/qt/playblend/main.cpp#L232">here</a>.<br />
<br />
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-17451806390470389682011-01-30T13:03:00.000-08:002012-03-20T14:18:46.866-07:00Cel shading, toon shading, silhouette extraction,... how to make the scene look better<div dir="ltr" style="text-align: left;" trbidi="on">
So far, it was about porting ReadBlend (Bullet physics world populated from .blend file) <a href="http://opengles20study.blogspot.com/2010/12/rendering-blender-model-in-opengl-es-20.html">from OpenGL ES 1.1 (fixed pipeline) to OpenGL ES 2.0 (GLSL)</a>, use example toon shader (blue-white-black color palette), and make camera slerp, lerp and follow the ball few metres behind it while ball is accelerating down the hill.<br />
<div>
I'd like to learn how to make it look better. </div>
<div>
<br /></div>
<div>
First approach would be to use texture in Toon shader.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAD0BrgGEmRID9RtJIaev6jvQ75PfDhwrgXgcXnnjn5jh3ASQ6ey6VNeqL-3nnhrsNaav3kRmb4_5G8BXyoIq398UnqPfGyR6TNWzq6_s5SCg_iKUAqK3iBO_Gdd68UipHZboLAdhs2l4o/s1600/CM+Capture+2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAD0BrgGEmRID9RtJIaev6jvQ75PfDhwrgXgcXnnjn5jh3ASQ6ey6VNeqL-3nnhrsNaav3kRmb4_5G8BXyoIq398UnqPfGyR6TNWzq6_s5SCg_iKUAqK3iBO_Gdd68UipHZboLAdhs2l4o/s320/CM+Capture+2.png" width="214" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbSPrgcqvax3U2hVqyqqj3R5shVMcNQN_AoAyF-V2k3Mc-3RlG_n8sUgpfCiaQoFokIMXhVxtGMryB_PTvfcpNl-ZDY6Ou2xIeO1ZOE2gDkbZcuhyphenhyphenyk1BslZJSE5wUQM4cTVU-Q9yt6SsG/s1600/CM+Capture+4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbSPrgcqvax3U2hVqyqqj3R5shVMcNQN_AoAyF-V2k3Mc-3RlG_n8sUgpfCiaQoFokIMXhVxtGMryB_PTvfcpNl-ZDY6Ou2xIeO1ZOE2gDkbZcuhyphenhyphenyk1BslZJSE5wUQM4cTVU-Q9yt6SsG/s320/CM+Capture+4.png" width="214" /></a></div>
<br />
<div>
Left side shows the original, with white <a href="http://en.wikipedia.org/wiki/Phong_shading">phong reflection</a>. On the right hand side is,... I think that silhouette looks nice but the texture is wrong for cartoon look. Phong doesn't look like phong anymore. Patch implementing described <a href="https://github.com/astoj/astoj_oolongengine/commit/79ed8fb83f7e6c8c83e6b3aeb6360d974232bbdc#L0R26">change to Toon.frag is here</a>. Might help... few lines of code copied from Texture.frag.<br />
<br />
I've simplified ReadBlend scene textures to be more cartoon-like, and modified Toon.frag to make silhouette have the same color as the area inside, just with darker tone. IMHO it looks better then blue white black scene from previous posts:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/BN__kLS-TGg/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/BN__kLS-TGg?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="320" height="266" src="http://www.youtube.com/v/BN__kLS-TGg?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
<br /></div>
<div>
<br />
<div>
<br />
Without geometry shaders or expensive analysis of faces on CPU side, I still need to work on finding proper technique. There are other things that I need to figure out first, so ... coming back to this later.<br />
Geometry shaders are not supported on iOS (or other OpenGL ES 2.0 implementations). I understand that hardware supports it (PowerVR SGX and SGXMP) but the feature is available only through DirectX Shader Model. Anyway, for other techniques on OpenGL ES, or for those who don't plan to use OpenGL ES (...can use geometry shaders) and are visiting this page, recommend to read <a href="http://prideout.net/blog/?p=54" target="_blank">The Little Grasshopper post about silhouette extraction</a>.<br />
<br />
Update on 27.02.2012 - found nice example <a href="http://www.imgtec.com/powervr/insider/powervr-sdk-docs.asp" target="_blank">in PowerVR documentation (document titled Edge Detection Training Course)</a>, based on edge detection postprocessing - render scene in first pass, detect and draw edges on GPU in second pass. I plan to give it a try soon. Note that edge detection is not slowish color comparison to surrounding texels; instead it packs objectId to alpha layer and postprocessing in second pass compares only with 2 surrounding texels. On iPhone 3GS this runs 60fps.<br />
<br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18px;">Applied this (using alpha channel to hold info about object) and got nice results (video presents edge size changing 1 -> 2 -> 3 pixels wide) </span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18px;"><br /></span></div>
<div class="separator" style="clear: both; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; text-align: center;">
<br /><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='480' height='400' src='https://www.youtube.com/embed/cJ8dYn4M7Yo?feature=player_embedded' frameborder='0'></iframe></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18px;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18px;"><br /></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: x-small;"><span class="Apple-style-span" style="line-height: 18px;">I plan to experiment using normals instead of object ids, to get internal edges too, not only silhouettes, and post full code then.</span></span></div>
<div>
<span class="Apple-style-span" style="color: #222222; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: x-small;"><span class="Apple-style-span" style="line-height: 18px;"><br /></span></span></div>
<br />
<br /></div>
</div>
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-70214238258232127382011-01-22T12:25:00.000-08:002011-11-09T10:10:22.224-08:00Camera movement animation, slerp and lerp interpolation, moving camera through 3D world<div dir="ltr" style="text-align: left;" trbidi="on">Code for the example is in this <a href="https://github.com/astoj/astoj_oolongengine/commit/fdb65cb373da02b08d9d04785202bba28e77aa77">commit</a>.<br />
<br />
Here, I wanted to make camera in example from the <a href="http://opengles20study.blogspot.com/2010/12/rendering-blender-model-in-opengl-es-20.html">first post</a> (where camera is static always displaying the scene from the same position) to move; initially to rotate around the scene from initial position to position right behind the ball. Once it is positioned behind the ball, start the Physics scene step simulation - ball would start rolling towards the target and camera would follow the ball down the hill.<br />
<br />
Check the video to see how it looks - I'll prepare the video and then explain the details.<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/RzqYh5_J-Nk/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/RzqYh5_J-Nk?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" /><param name="bgcolor" value="#FFFFFF" /><embed width="320" height="266" src="http://www.youtube.com/v/RzqYh5_J-Nk?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div><div class="" style="clear: both; text-align: left;"><br />
For this example, key points (in 3d space) that define camera trajectory and orientation are:</div><div class="" style="clear: both; text-align: left;">1) initial camera position as defined in .blend file is looking at the scene from left side</div><div class="separator" style="clear: both; text-align: left;">2) just behind the ball, looking at the direction of target</div><div class="" style="clear: both; text-align: left;">3) target (I just picked one of the cubes on the scene - <span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono',Courier,monospace; font-size: 12px; line-height: 17px; white-space: pre;">MECube.013</span>). <br />
<br />
</div><div class="separator" style="clear: both; text-align: left;"><b>Camera position animation from 1 - 2 </b>is done with</div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: helvetica,arial,freesans,clean,sans-serif; font-size: 11px; line-height: 14px;"></span></div><pre style="font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font-size: 12px; font: normal normal normal 12px/normal Monaco, 'Courier New', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; line-height: 1.4em; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><div class="line" id="LC306" style="line-height: 1.4em; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 1em; padding-right: 0px; padding-top: 0px;"><span class="k" style="line-height: 1.4em; margin: 0px; padding: 0px;">if</span> <span class="p" style="line-height: 1.4em; margin: 0px; padding: 0px;">(</span><span class="n" style="line-height: 1.4em; margin: 0px; padding: 0px;">cameraAnimation</span><span class="p" style="line-height: 1.4em; margin: 0px; padding: 0px;">)</span></div><div class="line" id="LC307" style="line-height: 1.4em; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 1em; padding-right: 0px; padding-top: 0px;"><span class="n" style="line-height: 1.4em; margin: 0px; padding: 0px;"> cameraAnimation</span><span class="o" style="font-weight: bold; line-height: 1.4em; margin: 0px; padding: 0px;">-></span><span class="n" style="line-height: 1.4em; margin: 0px; padding: 0px;">step</span><span class="p" style="line-height: 1.4em; margin: 0px; padding: 0px;">();</span></div><div class="line" id="LC307" style="line-height: 1.4em; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 1em; padding-right: 0px; padding-top: 0px;"></div></pre> where cameraAnimation is instance of <a href="https://github.com/astoj/astoj_oolongengine/commit/f83831797f184af389cdb48cc4972c55bff024bb">TransformInterpolator</a>; Interpolating - animating values from one matrix to another using slerp for rotation and lerp for translation. TransformInterpolator combines MATRIX lerp and slerp, provides convenience step() and finished() to model animation interpolation.<br />
<br />
To get the slerp working - there are 2 slerp implementations is oolongengine, I'm using the one from Math/Matrix.cpp. Could have picked also BulletPhysics but I did not want to convert MATRIX -> btTransform -> bqQuaternion -> do the slerp (then back from quaternion to MATRIX) -> btTransform -> float[]. Reason I'm planning to use Math/Matrix.cpp code is implementation of <a href="https://github.com/astoj/astoj_oolongengine/blob/master/Math/neonmath/neon_matrix_impl.cpp#L33">Neon arm7 matrix multiply</a>. Quite possible that I could be wrong, that there is no difference in performances between MATRIX's and btTransform's matrix multiplication...<br />
<br />
First problem I faced, and it took me some time to debug - refreshing meanwhile knowledge about matrix scaling, velocity and determinant - was that converting MATRIX from <a href="https://github.com/astoj/astoj_oolongengine/commit/fdb65cb373da02b08d9d04785202bba28e77aa77#L4R400">MatrixLookAtLH</a> to quaternion and back to matrix resulted with scaled matrix. Fix was to <a href="https://github.com/astoj/astoj_oolongengine/commit/fdb65cb373da02b08d9d04785202bba28e77aa77#L4R401">normalize vectors </a>after multiplying in MatrixLookAt - not before multiplication. After this patch, it started to work. I added also matrix to quaternion <a href="https://github.com/astoj/astoj_oolongengine/commit/fdb65cb373da02b08d9d04785202bba28e77aa77#L4R916">here</a>.<br />
<br />
<b>Following the ball</b><br />
Once the camera reach position 2, cameraAnimation->finished() returns true and we start stepping btDiscreteDynamicsWorld, the ball goes down the hill and camera is following it, looking toward the target.<br />
Figured out that, if I would like to slow down btDiscreteDynamicsWorld simulation and make camera fly around the world in slow motion, just need to supply value for sceneSlowdownWhileAnimatingCamera lower then one, e.g. 0.5, to btDiscreteDynamicsWorld->(sceneSlowdownWhileAnimatingCamera * 1./60.).<span class="Apple-style-span" style="line-height: 17px; white-space: pre;"><span class="Apple-style-span" style="font-family: inherit;"> </span></span><br />
<span class="Apple-style-span" style="line-height: 17px; white-space: pre;"><span class="Apple-style-span" style="font-family: inherit;">Anyway, let's see first how to get that toon shader and scene look better.</span></span><br />
<div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono',Courier,monospace; font-size: 12px; line-height: 17px; white-space: pre;"><br />
</span></div></div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0tag:blogger.com,1999:blog-8462305102076756733.post-54922257944810903452010-12-26T12:04:00.000-08:002011-01-30T13:18:54.828-08:00Rendering Blender model in OpenGL ES 2.0 - porting OpenGL ES 1.1 (fixed) to 2.0 (programmable GLSL) pipelineThe code I was able to find in SIO2 and oolongengine (blender reader copied from gamekit) was rendering Blender models using fixed pipeline. I needed to get cartoonlike rendering, so decided to go with OpenGL ES 2.0 and GLSL.<br />
Oolongengine (3Dlabs Inc) includes this <a href="http://code.google.com/p/oolongengine/source/browse/#svn%2Ftrunk%2FOolong%20Engine2%2FExamples%2FRenderer%2FTutorials%2F08%20Shaders%20Demo%20(3DShaders.com)">OpenGL ES 2.0 example - rendering rotating teapot:</a><br />
<div align="center"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/RPnhE3cME9Y?feature=player_embedded' frameborder='0'></iframe> </div><br />
The shader on video is simple <a href="http://code.google.com/p/oolongengine/source/browse/trunk/Oolong%20Engine2/Examples/Renderer/Tutorials/08%20Shaders%20Demo%20(3DShaders.com)/Classes/Shader/3DShaders.com/Toon.frag?r=214">toon shader</a>. Decided to use it when implementing this example.<br />
<br />
Couldn't find an OpenGL ES 2.0 example that would render 3d model loaded from .blend or .pod file - found only examples with OpenGL ES 1.1 code. One of those OpenGL ES 1.1 examples, <a href="http://code.google.com/p/oolongengine/source/browse/#svn%2Ftrunk%2FOolong%20Engine2%2FExamples%2FDemos%2FReadBlend">ReadBlend from oolongengine</a> looks like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/_5-u4BxPWh8/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/_5-u4BxPWh8?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" /><param name="bgcolor" value="#FFFFFF" /><embed width="320" height="266" src="http://www.youtube.com/v/_5-u4BxPWh8?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div><br />
I'll try to explain the approach to combine them:<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/d4ObXjpnTIY?feature=player_embedded' frameborder='0'></iframe></div><br />
<br />
Code is available <a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad">here</a>.<br />
<br />
Class Piper, with PiperGL11 and PiperGL20 subclasses is a simple <i>work in progress</i>, intended to abstract differences in fixed and rendering code to one place. Now, it includes push&pop&set&multMatrix operations.<br />
e.g. in the code, it was just required to replace ES 1.1 call <a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad#L3L331">glMultMatrix()</a> with call to <a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad#L3R335">Piper::instance()->multMatrix()</a> and that would bring support for both OpenGL ES 1.1 and OpenGL ES 2.0.<br />
Decided not to use pattern to first select mode with glMatrixMode() and then do matrix operation, as it looked cleaner to supply matrix mode for each operation. Of course, in PiperGL11 <a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad#L11R34">avoid redundant setting the same mode</a> with setMatrixMode.<br />
<br />
For OpenGL ES 2.0 version, it is needed to <a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad#L13R152">set the values for uniforms</a> just before the call to glDrawElements. I guess it is important to optimize this, and handle the case when there were no change in uniforms between consecutive calls to <span class="Apple-style-span" style="line-height: 17px; white-space: pre;"><span class="Apple-style-span" style="font-family: inherit;">Piper::instance()->glDrawElements. <span class="Apple-style-span" style="font-family: inherit;">Then, </span></span></span><span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 17px; white-space: pre;"><span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="font-family: inherit;"><a href="https://github.com/astoj/astoj_oolongengine/commit/d45f339b408dfcbcdbb95b81b7ec2ea493e563ad#L13R125">setupChangedVariables</a> should be able to handle this.</span></span></span><span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 17px; white-space: pre;"> </span></span></span><br />
<br />
Plan to work on touch input and navigating through scene, maybe with camera behind the ball, check different versions of toon shaders I could find online,... figure out how to add some more color to scene - with color attribute array or textures and check if there is difference in performances... <br />
<br />
<br />
<br />
<br />
<div align="left" class="separator" style="clear: both; text-align: center;"><br />
</div><div align="left" class="separator" style="clear: both; text-align: center;"><br />
</div><div align="justify" class="separator" style="clear: both; text-align: center;"><br />
</div><div align="justify" class="separator" style="clear: both; text-align: center;"><br />
</div><div align="justify" class="separator" style="clear: both; text-align: center;"><br />
</div>astojiljhttp://www.blogger.com/profile/14674929109224892631noreply@blogger.com0