CPU optimization ================ Measuring performance ===================== We have to know where the "bottlenecks" are to know how to speed up our program. Bottlenecks are the slowest parts of the program that limit the rate that everything can progress. Focussing on bottlenecks allows us to concentrate our efforts on optimizing the areas which will give us the greatest speed improvement, instead of spending a lot of time optimizing functions that will lead to small performance improvements. For the CPU, the easiest way to identify bottlenecks is to use a profiler. CPU profilers ============= Profilers run alongside your program and take timing measurements to work out what proportion of time is spent in each function. The Godot IDE conveniently has a built-in profiler. It does not run every time you start your project: it must be manually started and stopped. This is because, like most profilers, recording these timing measurements can slow down your project significantly. After profiling, you can look back at the results for a frame. .. figure:: img/godot_profiler.png) .. figure:: img/godot_profiler.png) :alt: Screenshot of the Godot profiler Results of a profile of one of the demo projects. Note: We can see the cost of built-in processes such as physics and audio, as well as seeing the cost of our own scripting functions at the bottom. Time spent waiting for various built-in servers may not be counted in the profilers. This is a known bug. When a project is running slowly, you will often see an obvious function or process taking a lot more time than others. This is your primary bottleneck, and you can usually increase speed by optimizing this area. For more info about using Godot's built-in profiler, see `doc_debugger_panel`. External profilers ~~~~~~~~~~~~~~~~~~ Although the Godot IDE profiler is very convenient and useful, sometimes you need more power, and the ability to profile the Godot engine source code itself. You can use a number of third party profilers to do this including `Valgrind ( https://www.valgrind.org/ )`, `VerySleepy ( http://www.codersnotes.com/sleepy/ )`, `HotSpot ( https://github.com/KDAB/hotspot )`, `Visual Studio ( https://visualstudio.microsoft.com/ )` and `Intel VTune ( https://software.intel.com/content/www/us/en/develop/tools/vtune-profiler.html )`. Note: You will need to compile Godot from source to use a third-party profiler. This is required to obtain debugging symbols. You can also use a debug build, however, note that the results of profiling a debug build will be different to a release build, because debug builds are less optimized. Bottlenecks are often in a different place in debug builds, so you should profile release builds whenever possible. .. figure:: img/valgrind.png) :alt: Screenshot of Callgrind Example results from Callgrind, which is part of Valgrind. From the left, Callgrind is listing the percentage of time within a function and its children (Inclusive), the percentage of time spent within the function itself, excluding child functions (Self), the number of times the function is called, the function name, and the file or module. In this example, we can see nearly all time is spent under the `Main::iteration()` function. This is the master function in the Godot source code that is called repeatedly. It causes frames to be drawn, physics ticks to be simulated, and nodes and scripts to be updated. A large proportion of the time is spent in the functions to render a canvas (66%), because this example uses a 2D benchmark. Below this, we see that almost 50% of the time is spent outside Godot code in `libglapi` and `i965_dri` (the graphics driver). This tells us the a large proportion of CPU time is being spent in the graphics driver. This is actually an excellent example because, in an ideal world, only a very small proportion of time would be spent in the graphics driver. This is an indication that there is a problem with too much communication and work being done in the graphics API. This specific profiling led to the development of 2D batching, which greatly speeds up 2D rendering by reducing bottlenecks in this area. Manually timing functions ========================= Another handy technique, especially once you have identified the bottleneck using a profiler, is to manually time the function or area under test. The specifics vary depending on the language, but in GDScript, you would do the following: ``` var time_start = OS.get_ticks_usec() # Your function you want to time update_enemies() var time_end = OS.get_ticks_usec() print("update_enemies() took %d microseconds" % time_end - time_start) ``` When manually timing functions, it is usually a good idea to run the function many times (1,000 or more times), instead of just once (unless it is a very slow function). The reason for doing this is that timers often have limited accuracy. Moreover, CPUs will schedule processes in a haphazard manner. Therefore, an average over a series of runs is more accurate than a single measurement. As you attempt to optimize functions, be sure to either repeatedly profile or time them as you go. This will give you crucial feedback as to whether the optimization is working (or not). Caches ====== CPU caches are something else to be particularly aware of, especially when comparing timing results of two different versions of a function. The results can be highly dependent on whether the data is in the CPU cache or not. CPUs don't load data directly from the system RAM, even though it's huge in comparison to the CPU cache (several gigabytes instead of a few megabytes). This is because system RAM is very slow to access. Instead, CPUs load data from a smaller, faster bank of memory called cache. Loading data from cache is very fast, but every time you try and load a memory address that is not stored in cache, the cache must make a trip to main memory and slowly load in some data. This delay can result in the CPU sitting around idle for a long time, and is referred to as a "cache miss". This means that the first time you run a function, it may run slowly because the data is not in the CPU cache. The second and later times, it may run much faster because the data is in the cache. Due to this, always use averages when timing, and be aware of the effects of cache. Understanding caching is also crucial to CPU optimization. If you have an algorithm (routine) that loads small bits of data from randomly spread out areas of main memory, this can result in a lot of cache misses, a lot of the time, the CPU will be waiting around for data instead of doing any work. Instead, if you can make your data accesses localised, or even better, access memory in a linear fashion (like a continuous list), then the cache will work optimally and the CPU will be able to work as fast as possible. Godot usually takes care of such low-level details for you. For example, the Server APIs make sure data is optimized for caching already for things like rendering and physics. Still, you should be especially aware of caching when using `GDNative