6. 3D display of camera images

The camera is OpenCV, the display is OpenGL + glut

In the previous article, OpenCV was used for display, but using OpenGL + glut for display makes 3D handling easier. The program camera_3d_glut.cpp of OpenCV for camera input and OpenGL + glut for 3D output was prepared. It is best to start with an example that moves as few lines as possible. Since this program has only 113 lines, you can easily read the whole program. If you download makefile and put it in the same directory and set it as 

make glut
, the camera image will be displayed as a 3D space video.
  • q key: quit
  • h key: View from the more left
  • j key: View from more below
  • k key: View from more above
  • l key: View from the more right
  • f key: View from a distance
  • n key: View closer
Key operation is possible.

The camera is OpenCV, the display is OpenGL + glut + glew

The OpenGL + glut part of the first example is the old way of writing when the GPU was not used for drawing. This way of writing seems to be unusable. In the new writing method, drawing data generation and drawing are written separately. In the prepared camera_3d_glew.cpp , this separation is performed using GL_ARRAY_BUFFER. In the idle () function, drawing data is generated immediately after the camera input, and the drawing itself is done with glDrawArrays () in the display () function. The number of lines has increased to 147, but if you take the difference from camera_3d_glut.cpp with diff, you can see how it changed. If you make 

make glew
using the previous makefile, the camera image will be displayed as a 3D space plane as in the first example. The key operation is the same.

The camera is OpenCV and the display is OpenGL + glut + glew with CUDA

The last example is drawing data generation with GPU + CUDA. Recently, CUDA is a parallel processing language environment developed by NVIDIA Corporation, which became famous for AI. If your computer has an NVIDIA GPU, such as GTX660, GTX1080, etc., you won't have to use CUDA. A total of 195 lines of camera_3d_cuda.cu were prepared. This time is 

make cuda
. The function is the same as the previous two. If you take the difference with camera_3d_glew.cpp, you can see how it changed. The drawing part in the display () function has not changed at all. The buffer allocation method and the drawing data generation part are different. In the idle () function picture<<< ((XX * YY) / TD_NUM) +1, TD_NUM >>> (pos); Drawing data is generated. The FRM array is used to transfer data from the camera to the GPU side.

Summary

Once you know the last example, you can create various things depending on the idea. Once you have a working example that is compact and comprehensible, you can experiment with the features that OpenCV, OpenGL, and CUDA don't know to make your own. Also, if you move up to here, OpenCV, OpenGL, and CUDA will be installed properly.