The GameBench Unity Package

The GameBench Unity Package

Current version: v1.3.2

Table of contents

1. What is the GameBench Unity Package?
2. Getting Started
3. Configuration
4. Metrics
5. The API
6. Overhead
7. Known Issues
8. Uninstallation
9. Other game engines
10. Appium

1. What is the GameBench Unity Package?

The GameBench Unity Package enables the capture and remote analysis of performance data for games developed with the Unity game engine. Performance data is uploaded to GameBench servers and may be browsed in the GameBench Web Dashboard or accessed programmatically via the GameBench Session Data API.

Supported platforms

• iOS 10.0 or above
• Android 5.0 or above (armeabi-v7a and arm64-v8a)
• We’ve tested this package with many devices, see our device list.

Supported Unity versions

• Unity 2018, 2019, 2020, 2021+.

(NB: 2017 is unsupported because it doesn’t include the Unity Package Manager, but you can make it work by manually unpacking your GameBench.tgz and importing the contents to this specific location in your project: Assets/GameBench/package.)

• Unity Scripting Backend: IL2CPP and Mono
• Unity Scripting Runtime Version: All stable .NET versions

2. Getting Started

2.1 Installation

Once you’ve obtained the package file (GameBench.tgz) perform the following steps:

1. Open your game project in Unity.
2. Open the ‘Package Manager’ (which is found under ‘Window’ in Unity’s menu).
3. In Package Manager, click the ‘+’ button in the top left corner and select ‘Add package from tarball…’ :
   
4. In the file browser select your GameBench.tgz from wherever you saved it. Package installation will proceed automatically.
5. When the package installation completes you will be prompted to configure it with your GameBench account details.

Unity 2018: Tarball packages aren’t supported in Unity 2018 so you will need to unpack the .tgz first and use “Add package from disk…” and browse to the package.json in the package root.

Unity 2019 and later: After the above steps your project manifest (Packages/manifest.json) will reference GameBench.tgz with an absolute file path. We recommend you manually edit the manifest file and change the GameBench entry to use a relative path, as per Unity documentation.

2.2. Setting your account details

In the Unity Editor UI, select “GameBench -> Configure” in the app menu to open a configuration window where you can set your account details.

Account

• Upload URL - The URL to which session data will be uploaded. Corresponds to serverendpoint in the config file (see section 3).
  • GameBench Cloud users should use https://web.gamebench.net/
  • Private server users should use their own URL.
• Email - The email address or LDAP username associated with your GameBench account. We recommend using a dedicated email account to store GameBench sessions. Corresponds to emailaddress in the config file.
• Token - API token (a hex string). See the API token documentation for more details.

Other Settings

• Enable GameBench - Controls whether GameBench will be included in your project builds. Corresponds to sdkEnable in the config file.
• Disable automatic capture - Tick this to disable the automatic session, as you’d do if you’d rather capture sessions through the C# API.

Screenshots

Screenshots are supported at a range of sizes: 100%, 50%, 25%, 12.5%, and 6.75%. The Editor UI lets you define up to 4 different captures at repeating intervals of your choosing. Screenshots may also be captured in script.

3. Configuration File

GameBench config is stored as JSON in the project file ProjectSettings/GameBench.json (NB: older versions of this SDK stored this as an asset at Assets/Resources/GameBenchSDK/SDKConfig.txt). To ensure GameBench’s impact on launch time is kept at effectively zero this JSON is converted into native source code declarations at build time.

It is possible to build your project without GameBench.json in which case configuration must be done at runtime, either by using the in-app C# API or - in the case of Appium testing - passing the config at launch time over the WebDriver interface (see section 10).

3.1. GameBench.json

{                                                     // Other names:
    "serverendpoint": "https://web.gamebench.net/",   // "url"
    "emailaddress": "me@company.com",                 // "email"
    "sdktoken": "0123456789ABCDEF0123456789ABCDEF",   // "token"
    "sdkAPIControlEnable": true,                      // "autoSession"
    "verboseSDK":true,                                // "verboseLog"
    "markSceneChanges":true,
    "sdkEnable": true,

    "metrics": 
    {                  // Screenshots
        "FPS": 1.0,    "SS0": 0.0,  // Full-size   (100%)
        "CPU": 1.0,    "SS1": 0.0,  // ¹⁄₂ size    (50%)
        "GPU": 1.0,    "SS2": 0.0,  // ¹⁄₄ size    (25%)
        "MEM": 4.0,    "SS3": 0.0,  // ¹⁄₈ size    (12.5%)
        "POW": 3.0,    "SS4": 0.0,  // ¹⁄₁₆ size   (6.75%)
        "BAT": 30.0,
        "NET": 30.0,
    },

    "tags": "foo=bar,baz=bat"
}

serverendpoint / url

The URL to which session data will be uploaded. - GameBench Cloud users should use https://web.gamebench.net/ - Private server users should use their own URL.

emailaddress / email

The email address or LDAP username associated with your GameBench account. We recommend using a dedicated email account to store GameBench sessions.

sdktoken / token

API token (a hex string). See the API token documentation for more details.

sdkAPIControlEnable / autoSession

These control whether the ‘automatic session’ is enabled or not (note that the alt name autoSession is the logical inverse of sdkAPIControlEnable). If it’s enabled then the automatic session will start when the app launches and will stop and upload when the app is backgrounded.

verboseSDK / verboseLog

This boolean flag is useful for troubleshooting, if set to ‘true’ GameBench will write events to the system log.

markSceneChanges

When set, scene changes will be automatically recorded as marker events. Default is ‘true’.

sdkEnable

If set false then Unity builds will not include the GameBench native library plugin and the effect is almost identical to uninstalling the package.

metrics

This is a map of metric names to their collection intervals, in seconds. Note that screenshots intervals are also stored here.

tags

Tags are an unordered set of comma-delimited key=value string pairs that are useful for identifying sessions or groups of sessions in the Analysis area of the Web Dashboard.

3.3. Google Identity-Aware Proxy (IAP) Configuration

If your GameBench server is behind IAP on Google Cloud then this SDK can support that. You will need to create a ‘service account’ in Google Cloud dashboard (if you don’t already have one), and then download the JSON config file for it and simply copy it into your Unity project at ProjectSettings/GoogleIAP.json. The credentials in it will be automatically used when connecting to your GameBench server.

IMPORTANT: You must edit the JSON and change the client_id value to the one that appears in the dashboard, which is a wholly different kind of client id.

4. Metrics

GameBench captures the following measurements at settable intervals:

5. The API

GameBench can be used without writing any code at all. By default, performance data will be automatically captured from app launch until the app is backgrounded at which point the captured data will be uploaded. But if you need finer control over data capture - to limit it to particular scenes or events in your game, say - then GameBench offers full control through an API.

NB: Before using the API it is strongly recommended that you disable automatic capture via the option in the configuration UI (pictured above).

All APIs are static methods of the Gamebench class in the GamebenchLib.Runtime namespace. You do not need to instantiate anything.

5.1 Sessions

5.2 Metric capture

All metrics may be captured at set intervals and/or with the CaptureNow() method.

Note that screenshots are a type of metric (see the SS values in the MetricType enum). To capture a screenshot at 50% the screen resolution you could write Gamebench.CaptureNow(MetricType.SS1).

Metric capture has very low overhead and is asynchronous, it should not affect your app’s observed performance.

5.3 Markers

To isolate specific areas of gameplay such as levels or battles, GameBench provides markers functionality.. For example, if you want to isolate performance data during a particular game level, you can set “in” marker when the level begins and “out” marker when the level is completed.

NB: Markers for scene changes are recorded automatically by default.

5.4 Tags

5.5 Configuration overrides

All other configuration values are also settable in code. Note that these values set with these APIs are transient and do not persist beyond process exit.

Where ConfigItem is a value from the following enum in the same namespace as Gamebench:

    public enum ConfigItem {
        UploadUrl,        // string
        UploadEmail,      // string
        UploadToken,      // string
        AutoSession,      // bool
        MarkSceneChanges, // bool
        VerboseLogging,   // bool
    }

6. Overhead

6.1 Executable size

GameBench will add around 0.9MB to your Android .apk and 0.6MB to your iOS .ipa (or just 0.3MB if you only build arm64).

6.2 RAM usage

GameBench performs very little dynamic allocation and it’s impact on RAM usage is reasonably constant at around ~200KB on both iOS and Android.

6.3 CPU and Power usage

To measure the impact of GameBench on CPU and Power usage we profiled game demos on common devices to measure the difference once GameBench was enabled. Each game demo required no user input and therefore performed the same path each time, each ran a minimum of 3 times and for the same duration each time.

CPU

Power

Storage and upload size overhead - 10Kb for 10 mins session

6.4 Screenshots

Screenshots can have a significant impact on session data sizes and upload times so GameBench screenshots use lossy JPEG compression to minimize this. For a typical smartphone with a ~3 megapixel screen a full-size capture will be saved at 50% JPEG quality and occupy roughly 100KB of storage. If you need very frequent screenshots we recommend using one of the reduced size capture options which need much less space.

7. Known Issues

Screenshot Stalls

A tiny minority of old phones running Android 5.0 will visibly stall when a screenshot is taken. The Samsung Galaxy S4 I-9500 running 5.0.1 in particular is especially bad for this. A scheduled update will work around the bad GL driver behaviour that causes this.

8. Uninstallation

To uninstall:

1. Open the Package Manager window in Unity.
2. Set the ‘Packages’ filter at the top of the window to ‘In Project’.
3. Select GameBench from the package list.
4. Click the “Remove” button in the bottom right corner. Uninstall will proceed automatically.

9. Other game engines

GameBench may also be integrated with non-Unity projects. All that is needed is to download the native libraries, add them to your Android Studio or XCode project and write a very small amount of C initialization code.

For a complete example of this refer to the GameBench-native.zip download in the portal which contains full instructions in it’s README.

10. Appium

GameBench can be controlled from Appium test scripts. Support exists for configuration, starting and stopping sessions, and recording markers. Please refer to the Extras~ folder within the package for documentation and examples in C#, JS, and Java.