Using short value in vertex array instead of float

I've been trying to squeeze more FPS from the rendering part of Smart Brick. I came across an article which suggests using GL_SHORT instead of GL_FLOAT in vertex array to boost performance:

http://stackoverflow.com/questions/1762866/using-gl-short-instead-of-gl-float-in-an-opengl-es-vertex-array

The solution quite makes sense. Basically you use short value to replace float values; to compensate the precision loss, a pair of additional glScalef and glTranslatef is required per-model.

A reply attached at the end of the post suggests that on iPhone it would be 30% faster. However, this is not the case for Smart Brick; not because the article is wrong, but I think the performance increment depends on actual scenarios. Here is my guess:

If your model contains large amounts of vertices the solution may help a lot. You get big computational savings for using short instead of float, with the price of only one additional glScale and glTranslate calls respectively.

However, it does not help Smart Brick, because Smart Brick renders large amounts of models, each of them contains only few vertices. This way, the savings on float vertices are probably offset by glScale and glTranslate calls, because they are called once per-model, and they are called a lot when there are a lot of models.

How to select 3D object from cursor/touch in OpenGL ES

When writing 3D games there is a requirement quite common (forget about OpenGL ES at the moment):

How to select an 3D object in the scene, by mouse/touch?

This problem is not so trivial as it seems. Actually the more you think about it, the more you will find it is not so easy to solve.

Basically, we are trying to map the windows coordinate back to the 3D coordinate. If you understand how an object in OpenGL coordinate is projected onto your 2D display:

screen_vector = ((3D_vector_4x1) * model_view_matrix_4x4) * projection_matrix_4x4;

All we need to do is do the reverse, multiply the screen_vector with the reverse of model view/projection matrices:

3D_vector_4x1 = (screen_vector * (projection_matrix_4x4_inv)) * model_view_matrix_4x4_inv;

Actually, in desktop version of OpenGL, you don't have to do the calculation yourself. There is the mighty gluUnproject that handles it for you.

But there is still a problem. Since your LCD screen is 2D, you only get the x,y component of screen_vector. Where is z? z is actually the depth component and you can find it in the depth buffer. In desktop version of OpenGL, you can call glReadPixels with GL_DEPTH_COMPONENT flag. Of course it is not the full story: the depth buffer only stores pixel depth values that are minimal (unless you change the OpenGL depth function), when transformed back to 3D_vector_4x1 you only get the nearest object. Usually it is desired, but if your wanted object is painted behind other transparent polygons, there is a little bit of problem - but hey it's not a big deal.

But when comes to OpenGL ES, there is a big problem implementing all these:
1. You don't get GLU package. So there is no gluUnproject.
2. You don't have access to the depth buffer. (No GL_DEPTH_COMPONENT for you!)

So the previous solution won't work!
Face it, in OpenGL ES it is a one-way road: you can get windows coordinate from a 3D coordinate, but there is no other way around. Then how to solve the problem?

Well... brutal force always works. First, you define what object is selectable in the game world. Second, you calculate the window coordinate of that object during OpenGL paint, keeping it updated. Last, when a mouse click/touch happens, compare the event windows coordinate to all existing window coordinates that updated during OpenGL paint, and pick one that is nearest. Of course, it is not universal and not all objects in the world can be selected; but hey, do you really need that?

Or you can use ray-tracing.