The GameBench Unity Package

Latest Version: 0.8.2
Status: This version should be considered Beta quality and should not be included into release or release candidate games.

Table of contents

  1. What is the GameBench Unity Package?
  2. Installation
  3. Configuration
  4. Metrics
  5. The API
  6. Overhead
  7. Known Issues
  8. Uninstallation

1. What is the GameBench Unity Package?

The GameBench Unity Package enables the capture and remote analysis of performance data for games developed with the Unity game engine. Performance data is uploaded to GameBench servers and may be browsed in the GameBench Web Dashboard or accessed programmatically via the GameBench Session Data API.

Supported platforms

  • iOS 10.0 or above
  • Android 5.0 or above (armeabi-v7a and arm64-v8a)
  • We’ve tested this package with many devices, see our device list.

Supported Unity versions

  • Unity 2018, 2019, 2020. (NB: 2017 is unsupported but should be usable with special install instructions)
  • Unity Scripting Backend: IL2CPP and Mono
  • Unity Scripting Runtime Version: All stable .NET versions

2. Installation


IMPORTANT

If upgrading from version 0.6.1 or older, you should first manually remove all GameBench files from your project assets. Versions before 0.7 were in the older ‘asset package’ format where GameBench code and plugins were imported/copied into your project. Build 0.7 onwards uses the newer Package Manager package form.


Once you’ve obtained the package file (GameBench.tgz) perform the following steps:

  1. Open your game project in Unity.
  2. Open the ‘Package Manager’ (which is found under ‘Window’ in Unity’s menu).
  3. In Package Manager, click the ‘+’ button in the top left corner and select ‘Add package from tarball…’ :
    sdk_package_manager
  4. In the file browser select your GameBench.tgz from wherever you saved it. Package installation will proceed automatically.
  5. When the package installation completes you will be prompted to configure it with your GameBench account details.

Unity 2018: Tarball packages aren’t supported in Unity 2018 so you will need to unpack the .tgz first and use “Add package from disk…” and browse to the package.json in the package root.

Unity 2019 and later: After the above steps your project manifest (Packages/manifest.json) will reference GameBench.tgz with an absolute file path. We recommend you manually edit the manifest file and change the GameBench entry to use a relative path, as per Unity documentation.

3. Configuration

3.1. In the Editor

GameBench must be configured with some account details before use. This is easily done with the Editor UI which is accessed from “GameBench -> Configure” in the Unity menu:

sdk_config

Account
  • Upload URL - Base URL of the server where performance data will be stored.
  • Email - We recommend using a dedicated email account to store GameBench sessions.
  • Token - API token (a hex string). See the API token documentation for more details.
Other Settings
  • Enable GameBench - Controls whether GameBench will be included in your project builds.
  • Disable automatic capture - By default sessions are captured automatically, meaning a session starts when the app launches and that session is stopped and uploaded when the app is backgrounded. If you prefer to capture sessions through the C# API you can use this option to disable the automatic session capture.
Screenshots

Screenshots are supported at a range of sizes: 100%, 50%, 25%, 12.5%, and 6.75%. The Editor UI lets you define up to 4 different captures at repeating intervals of your choosing. Screenshots may also be captured in script.

3.2. Advanced Configuration

Not all GameBench config is exposed in the Editor UI. The full configuration is stored as JSON in the file Assets/Resources/GameBenchSDK/SDKConfig.txt. Here are the additional values that may be set using your favourite text editor:

{
    "markSceneChanges":true,
    "verboseSDK":true,

    "metrics": 
    {                  // Screenshots
        "FPS": 1.0,    "SS0": 0.0,  // Full-size   (100%)
        "CPU": 1.0,    "SS1": 0.0,  // ¹⁄₂ size    (50%)
        "GPU": 1.0,    "SS2": 0.0,  // ¹⁄₄ size    (25%)
        "MEM": 4.0,    "SS3": 0.0,  // ¹⁄₈ size    (12.5%)
        "POW": 3.0,    "SS4": 0.0,  // ¹⁄₁₆ size   (6.75%)
        "BAT": 30.0,
        "NET": 30.0,
    },

    "tags": "foo=bar,baz=bat"
}
markSceneChanges

When set, scene changes will be automatically recorded as marker events. Default is ‘true’.

verboseSDK

This boolean flag is useful for troubleshooting, set it to ‘true’ for GameBench to log what it’s doing.

metrics

This is a map of metric names to their collection intervals, in seconds. Note that screenshots intervals are also stored here.

tags

Tags are an unordered set of comma-delimited key=value string pairs that are useful for identifying sessions or groups of sessions in the Analysis area of the Web Dashboard.

4. Metrics

GameBench captures the following measurements at settable intervals:


Metric
Default
Interval

Notes
FPS 1s See FPS analysis.
CPU usage 1s See CPU metrics and analysis. Android CPU measurements are normalized and agree with the Android Studio profiler, while iOS CPU are not normalised and agree with the debug gauges in Xcode.
GPU usage 1s GameBench captures GPU usage on Android devices with Mali GPUs and iOS devices targeting Metal (OpenGL ES on iOS is not supported). List of GPU supported processors and devices.. More information on GPU analysis.
Memory 4s See Memory metrics and analysis.
Battery 30s See Battery metrics and analysis.
Network 30s See Network metrics and analysis.

5. The API

GameBench can be used without writing any code at all. By default, performance data will be automatically captured from app launch until the app is backgrounded at which point the captured data will be uploaded. But if you need finer control over data capture - to limit it to particular scenes or events in your game, say - then GameBench offers full control through an API.

NB: Before using the API it is strongly recommended that you disable automatic capture via the option in the configuration UI (pictured above).

All APIs are static methods of the Gamebench class in the GamebenchLib.Runtime namespace. You do not need to instantiate anything.

5.1 Sessions

Method Notes
Gamebench.Start(string sessionTitle = null) Starts a new capture session, with an optional title.
Gamebench.Stop() Stops the current capture session.
Gamebench.Upload(UploadCallback) Uploads any outstanding sessions to the endpoint configured in the UI. After successful upload the session data will be removed from the device. Must be called without an active session.
Gamebench.Reset() Deletes all GameBench data on the device. Must be called without an active session.

5.2 Metric capture

All metrics may be captured at set intervals and/or with the CaptureNow() method.

Method Notes
Gamebench.SetCaptureInterval(MetricType, float) Enable and set the capture interval in seconds for the given metric type. Can be called with or without an active capture session. Use an interval of 0 or less to disable
Gamebench.CaptureNow(MetricType) Capture and write out the given metric now.

Note that screenshots are a type of metric (see the SS values in the MetricType enum). To capture a screenshot at 50% the screen resolution you could write Gamebench.CaptureNow(MetricType.SS1).

Metric capture has very low overhead and is asynchronous, it should not affect your app’s observed performance.

5.3 Markers

To isolate specific areas of gameplay such as levels or battles, GameBench provides markers functionality.. For example, if you want to isolate performance data during a particular game level, you can set “in” marker when the level begins and “out” marker when the level is completed.

Method Notes
Gamebench.MarkStart(string) Record a ‘start’ marker with the given name.
Gamebench.MarkStop(string) Record a ‘stop’ marker with the given name.
Gamebench.MarkLaunchComplete() Record a special ‘launch complete’ marker.

NB: Markers for scene changes are recorded automatically by default.

5.4 Tags

Method Notes
Gamebench.SetTag(string, string) Set a tag in the tags collection
Gamebench.RemoveTag(string) Remove a tag from the tags collection

5.5 Configuration overrides

All other configuration values are also settable in code. Note that these values set with these APIs are transient and do not persist beyond process exit.

Method Notes
Gamebench.SetConfigItem(ConfigItem, string | bool) Set a configuration value.

Where ConfigItem is a value from the following enum in the same namespace as Gamebench:

    public enum ConfigItem {
        UploadUrl,        // string
        UploadEmail,      // string
        UploadToken,      // string
        AutoSession,      // bool
        MarkSceneChanges, // bool
        VerboseLogging,   // bool
    }

6. Overhead

6.1 Executable size

GameBench will add around 0.9MB to your Android .apk and 0.6MB to your iOS .ipa (or just 0.3MB if you only build arm64).

6.2 RAM usage

GameBench performs very little dynamic allocation and it’s impact on RAM usage is reasonably constant at around ~200KB on both iOS and Android.

6.3 CPU and Power usage

To measure the impact of GameBench on CPU and Power usage we profiled game demos on common devices to measure the difference once GameBench was enabled. Each game demo required no user input and therefore performed the same path each time, each ran a minimum of 3 times and for the same duration each time.

CPU
iPhone 11 Samsung Galaxy S20 Ultra
CPU 0.17% 0.12%
Power
Device Without SDK (3 mins) With SDK
Huawei P40 Pro (Run1) 247.6 248
Huawei P40 Pro (Run2) 247.5 248.8
Huawei P40 Pro (Run3) 247.5 248.6
Avg - 247.5 mA Avg - 248.5 (~1 mA overhead)
iPhone 11 Pro Max (Run1) 191 191.87
iPhone 11 Pro Max (Run2) 199 199.28
iPhone 11 Pro Max (Run3) 190.6 191
Avg - 193.5 mA Avg - 194.05 (~0.6 mA overhead)

Storage and upload size overhead - 10Kb for 10 mins session

6.4 Screenshots

Screenshots can have a significant impact on session data sizes and upload times so GameBench screenshots use lossy JPEG compression to minimize this. For a typical smartphone with a ~3 megapixel screen a full-size capture will be saved at 50% JPEG quality and occupy roughly 100KB of storage. If you need very frequent screenshots we recommend using one of the reduced size capture options which need much less space.

7. Known Issues

Screenshot Stalls

A tiny minority of old phones running Android 5.0 will visibly stall when a screenshot is taken. The Samsung Galaxy S4 I-9500 running 5.0.1 in particular is especially bad for this. We hope to work around the bad driver behaviour that causes this in a future release.

Random reboots

The Huawei Honor 9 has been observed to occasionally reboot itself - indicating a kernel panic - within a minute of capturing a session including screenshots. We speculate that a combination of rarely-exercised paths in the Mali GPU driver and Huawei’s custom optimization service are to blame. We are investigating this issue in the hope of finding a satisfactory workaround.

8. Uninstallation

To uninstall:

  1. Open the Package Manager window in Unity.
  2. Set the ‘Packages’ filter at the top of the window to ‘In Project’.
  3. Select GameBench from the package list.
  4. Click the “Remove” button in the bottom right corner. Uninstall will proceed automatically.