Sunday, January 30, 2011

Cel shading, toon shading, silhouette extraction,... how to make the scene look better

So far, it was about porting ReadBlend (Bullet physics world populated from .blend file) from OpenGL ES 1.1 (fixed pipeline) to OpenGL ES 2.0 (GLSL), use example toon shader (blue-white-black color palette), and make camera slerp, lerp and follow the ball few metres behind it while ball is accelerating down the hill.
I'd like to learn how to make it look better. 

First approach would be to use texture in Toon shader.

Left side shows the original, with white phong reflection. On the right hand side is,... I think that silhouette looks nice but the texture is wrong for cartoon look. Phong doesn't look like phong anymore. Patch implementing described change to Toon.frag is here. Might help... few lines of code copied from Texture.frag.

I've simplified ReadBlend scene textures to be more cartoon-like, and modified Toon.frag to make silhouette have the same color as the area inside, just with darker tone. IMHO it looks better then blue white black scene from previous posts:




Without geometry shaders or expensive analysis of faces on CPU side, I still need to work on finding proper technique. There are other things that I need to figure out first, so ... coming back to this later.
Geometry shaders are not supported on iOS (or other OpenGL ES 2.0 implementations). I understand that hardware supports it (PowerVR SGX and SGXMP) but the feature is available only through DirectX Shader Model. Anyway, for other techniques on OpenGL ES, or for those who don't plan to use OpenGL ES (...can use geometry shaders) and are visiting this page, recommend to read The Little Grasshopper post about silhouette extraction.

Update on 27.02.2012 - found nice example in PowerVR documentation (document titled Edge Detection Training Course), based on edge detection postprocessing - render scene in first pass, detect and draw edges on GPU in second pass. I plan to give it a try soon. Note that edge detection is not slowish color comparison to surrounding texels; instead it packs objectId to alpha layer and postprocessing in second pass compares only with 2 surrounding texels. On iPhone 3GS this runs 60fps.


Applied this (using alpha channel to hold info about object) and got nice results (video presents edge size changing 1 -> 2 -> 3 pixels wide) 




I plan to experiment using normals instead of object ids, to get internal edges too, not only silhouettes, and post full code then.



Saturday, January 22, 2011

Camera movement animation, slerp and lerp interpolation, moving camera through 3D world

Code for the example is in this commit.

Here, I wanted to make camera in example from the first post (where camera is static always displaying the scene from the same position) to move; initially to rotate around the scene from initial position to position right behind the ball. Once it is positioned behind the ball, start the Physics scene step simulation - ball would start rolling towards the target and camera would follow the ball down the hill.

Check the video to see how it looks - I'll prepare the video and then explain the details.


For this example, key points (in 3d space) that define camera trajectory and orientation are:
1) initial camera position as defined in .blend file is looking at the scene from left side
2) just behind the ball, looking at the direction of target
3) target (I just picked one of the cubes on the scene - MECube.013).

Camera position animation from 1 - 2 is done with
if (cameraAnimation)
cameraAnimation->step();
 where cameraAnimation is instance of TransformInterpolator; Interpolating - animating values from one matrix to another using slerp for rotation and lerp for translation. TransformInterpolator combines MATRIX lerp and slerp, provides convenience step() and finished() to model animation interpolation.

To get the slerp working - there are 2 slerp implementations is oolongengine, I'm using the one from Math/Matrix.cpp. Could have picked also BulletPhysics but I did not want to convert MATRIX -> btTransform -> bqQuaternion -> do the slerp (then back from quaternion to MATRIX) -> btTransform -> float[]. Reason I'm planning to use Math/Matrix.cpp code is implementation of Neon arm7 matrix multiply. Quite possible that I could be wrong, that there is no difference in performances between MATRIX's and btTransform's matrix multiplication...

First problem I faced, and it took me some time to debug - refreshing meanwhile knowledge about matrix scaling, velocity and determinant - was that converting MATRIX from MatrixLookAtLH to quaternion and back to matrix resulted with scaled matrix. Fix was to normalize vectors after multiplying in MatrixLookAt - not before multiplication. After this patch, it started to work. I added also matrix to quaternion here.

Following the ball
Once the camera reach position 2, cameraAnimation->finished() returns true and we start stepping btDiscreteDynamicsWorld, the ball goes down the hill and camera is following it, looking toward the target.
Figured out that, if I would like to slow down btDiscreteDynamicsWorld simulation and make camera fly around the world in slow motion, just need to supply value for sceneSlowdownWhileAnimatingCamera lower then one, e.g. 0.5, to btDiscreteDynamicsWorld->(sceneSlowdownWhileAnimatingCamera * 1./60.). 
Anyway, let's see first how to get that toon shader and scene look better.