Death to Box Selection

Ok good so now that you’ve hopefully been click-baited into here by the title, let’s answer some questions right up front. What is box selection and why does Ulf think it’s time to put it down? What would take its place?

Well, first off, when we’re talking about box selection from here on in, what I’m referring to specifically is the de facto method for selecting multiple units in PC real time strategy games. To wit: left-clicking and dragging to define the opposing corners of a rectangle, then, upon releasing the left-mouse button, finding and selecting any units within the projected area of said rectangle.

“I know what box selection is you dingus” you are saying, “because I am reading this on my computer and I can tab over to my desktop and do it right now.” You are right! (unless you are reading this on mobile, in which case you are lying)

Some History

Box selection came to RTS’s from desktops and file management systems, where it was quite at home and made total sense. Dragging a box around a set of elements on a grid is a perfectly sound and simple solution to wanting to group and manipulate square icons. The earliest RTS’s, along with most of the PC gaming world in the 90’s, were sprite-sheet based and so every object and entity existed as a box on a square grid (or, when things got fancy and implied perspective worked its way in, a squashed, rectangular grid). As such, modeling interface mechanisms off of file management systems made the most logical sense for the pioneers of the genre.

Graphically, 2D sprite engines make box selection nice and clean to implement as well, as screen space (where the user’s cursor and thus any box they drag with it exists) translates almost directly to game world space, with just a simple offset and bounds or really at most a uniform scale factor and possible rotation matrix transformation.

All well and good. But! As the years have gone by, and we’ve moved into the radical new world of three dimensional gaming, some interesting eccentricities have emerged, bringing us to our second and more important question: what are the issues with box selection and thusly why does Ulf think it’s time to investigate alternatives?

Some Problems

First, box selection is somewhat awkward in three dimensions. The standard bird’s eye view perspective camera most RTS’s rely on views a trapezoidal swath of the game world’s terrain. The further the camera angle is set from being perpendicular to the ground (that is, further from staring straight down), the more pronounced and extreme the projected shape becomes. Any screen space rectangle reflects this projection, translating to a world space trapezoid with a somewhat narrow base and somewhat wide top. This also has the result that a box made in the upper half of the screen covers more ground in the game world than the exact same box in the lower half. It doesn’t usually present itself as much of a problem, but does mean that it is slightly harder to select a large number of units towards the bottom of the screen, and is slightly harder to make a precise selection on units towards the top.

The much more pertinent issue, however, has actually been inherent to box selection from the start, but has been made more and more pronounced by the jump to 3D and the evolutions that have been made along the way there. That issue: things that aren’t orthogonally aligned to the screen. Box selection can only define rectangles whose edges are parallel to the screen’s. Once any group of things stops marching along perfectly aligned to the x or y axis, the bounds of the screen space box required to contain said group invariably also contains redundant area. That redundant area doesn’t necessarily have an increased time cost in making a selection action, but becomes a real problem if it contains anything the player doesn’t want to select. Once the orthogonal bounding box of a group of selectable units contains other undesired units, box selection requires multiple steps of action. The player must either make a series of additive selections or a fully bounded selection followed by a series of subtractions in order to exclude the units not wanted in a precise selection.

You’re undoubtedly familiar with this issue if you’ve ever moved a large number of photos from a family vacation album or cleaned old icons and files off of a desktop, but in those scenarios speed is hardly a factor. In real time strategy games, however, precise selection is a matter of issuing orders and controlling armies in a very time sensitive manner – as in, well, in real time.

This issue has gotten worse as RTS’s have gotten better. With the shift to 3D models and units ‘footprints’ becoming circles instead of squares, alongside advances in pathfinding and ‘clumping’ behavioral AI, units wind up more frequently hanging out near each other and more regularly distributing themselves into ideal damage-dealing formations during combat (which is almost always an arc with a radius equal to or less than their attack range). As we’ve found new and interesting ways within the genre to design abilities, create gameplay and allow counterplay, as we’ve created more unit archetypes that diverge more than simply having varied damage, range, health, etc., we’ve created more and more scenarios where targeted areas or unique attack patterns ask players to have specific units react in specific ways as quickly as possible during hectic engagements. As we’ve gotten better at utilizing space and designing competitive maps, we’ve created more and more common situations where battle lines are drawn along diagonals (in Starcraft 2, for example, many competitive maps have embedded conditional logic to ensure that player bases spawn in opposite corners at the start of the match) – the worst case possible for box selection.

Time to Experiment

Which brings us to our investigation of a potential alternative – what I’m going to tentatively be calling ‘Circle-Drag-Select’ – or just ‘Circle Select,’ as the antithesis to box select (‘Bubble Select’ was another consideration, but it kinda implied association with Bubble Sort which would be misleading). So what is it? In short: a cursor with an adjustable radius, in world space. If you’re familiar with Blender, then you might already know what we’re going for. If not, think: a circular paintbrush in Photoshop, but raycast from the cursor’s screen location to the world terrain, then drawn outwards as a sphere. When you left-click, it grabs every valid selectable unit within that radius. As you left-click and drag, each new unit that falls within the sphere’s path is added to the current selection. To select a diagonal line of units, for example, you take the same kind of action you would under box selection: left-clicking and dragging from one end of the group to the other, then releasing.

However, circle select’s flexibility opens up a number of advantages where box select struggles. A large group of units can be selected with a single left-click with a large radius. A precise number of units can be selected in any arbitrary formation by left-click and dragging a path over them with a small radius. Undesired units can be skirted around, avoiding compound actions in favor of longer, sweeping ones. By relying on only a single raycast, it is essentially a world-to-world solution, and alleviates some of the screen-to-world woes of box selection.

To the Laboratory!

I’ve whipped up a Unity demo of circle select, so let’s dig in to the implementation. To do a meaningful demonstration, I had to build a lightweight RTS as a framework, but since that’s not really our focus here, and it’s not particularly interesting, I won’t be breaking it down in detail. There are a number of functional methods I wrote for it that will come up in the code below, but they’re hopefully all descriptively named enough to be self-explanatory. So! First things first, we convert the cursor’s screen location to a point in world space by raycasting from the camera to the terrain. Unity’s built-in physics system and camera functions make this quite straightforward:

private Vector3 GetCursorPoint () {
  RaycastHit hit;
  int mask = LayerMask.GetMask("Terrain");
  Physics.Raycast(Camera.main.ScreenPointToRay(Input.mousePosition), out hit, Mathf.Infinity, mask);

  return hit.point;
}

With world coordinates known, we can then easily find what units are within a certain radius from that point with the following function:

public static Unit[] FindUnitsAtPoint (Vector3 point, float radius) {
  int mask = LayerMask.GetMask("Default");
  Collider[] colliders = Physics.OverlapSphere(point, radius, mask);

  Unit[] units = new Unit[0];
  foreach (Collider c in colliders) {
    Unit u = c.GetComponent<Unit>();
    if (u != null) {
      units = AppendToArray(units, u);
    }
  }

  return units;
}

The Physics.OverlapSphere() call is essentially the whole body of the function, the rest is just making sure to catch and extract references to the ‘Unit’ monobehavior components to pass back. Most of the rest of what we want to do then, can be accomplished by gathering, storing and manipulating arrays of Units returned via this FindUnitsAtPoint() function.

In the interest of putting together this demo quickly, I’ve simply written out the selection logic as a series of conditional statements within a Unity Update() call (the more elegant and robust solution would probably be to set up and utilize event listeners). The first thing we want to do is a bit of housekeeping, making sure to clear out references to any units who may have died in the previous frame so we don’t have null pointers in our arrays.

void Update () {
  //cleanup old units array
  selectedUnits = Unit.RemoveNullEntries(selectedUnits);
  hoveredUnits = Unit.RemoveNullEntries(hoveredUnits);
  for (int i=0; i<controlGroups.Length; i++) {
    controlGroups[i] = Unit.RemoveNullEntries(controlGroups[i]);
  }
  /*...*/

Next up, we want to adjust the selection radius. In looking into RTS control schemes, I found that many had room to re-map the generally useless camera-zoom-in of the mouse-wheel to our new input axis, so that’s what we’ll be using in this implementation. There’s a little bit of extra code here to allow for some input smoothing and elasticity. In short, the mouse-wheel drives a clamped target radius value that the actual selection radius chases.

  /*...*/
  //listen for mousewheel, update selection radius
  if (Input.GetButtonDown("ResetRadius")) {  //Has the middle mouse button been pressed?
    targetRadius = SettingsManager.defaultRadius;
  }
  else {
    if (SettingsManager.isInverted) {  //Inverted mousewheel scrolling
      targetRadius -= SettingsManager.mousewheelMultiplier * Input.GetAxis("AdjustRadius");
    }
    else {  //Mousewheel scrolling
      targetRadius += SettingsManager.mousewheelMultiplier * Input.GetAxis("AdjustRadius");
    }
    if (targetRadius < 0.0f) {
      if (targetRadius < -1.0f) {
        targetRadius = -1.0f;
      }
      else {
        targetRadius = Mathf.Lerp(targetRadius, 0.0f, SettingsManager.radiusAdjustmentSmoothing*Time.deltaTime);
      }
    }
    if (targetRadius > SettingsManager.maxSelectionRadius) {
      if (targetRadius > SettingsManager.maxSelectionRadius + 1.0f) {
        targetRadius = SettingsManager.maxSelectionRadius + 1.0f;
      }
      else {
        targetRadius = Mathf.Lerp(targetRadius, SettingsManager.maxSelectionRadius, SettingsManager.radiusAdjustmentSmoothing*Time.deltaTime);
      }
    }
  }
  if (selectionRadius != targetRadius) {
    selectionRadius = Mathf.Lerp(selectionRadius, targetRadius, SettingsManager.radiusAdjustmentSmoothing*Time.deltaTime);
    if (selectionRadius < 0.01f) {
      selectionRadius = 0.01f;
    }
    if (selectionRadius > SettingsManager.maxSelectionRadius) {
      selectionRadius = SettingsManager.maxSelectionRadius;
    }
  }
  /*...*/

After that, we use the functions from above to find the point in space the cursor is hovered over and find out what new units we may be interacting with in this frame (stored as ‘hoveredUnits‘).

  /*...*/
  //find the position in the world we are hovering the cursor over, along with any units we're hovering over
  foreach (Unit h in hoveredUnits) {
    h.BroadcastMessage("OnUnhover");
  }
  Vector3 cursorPoint = GetCursorPoint();
  hoveredUnits = Unit.FindUnitsAtPoint(cursorPoint, selectionRadius);
  foreach (Unit h in hoveredUnits) {
    h.BroadcastMessage("OnHover");
  }
  /*...*/

The rest of the bulk of the work we do in Update() is the overall selection and order issuing logic. We ask the Unity Input module if the left mouse button has been pressed, is being held down, or has been released this frame, then work out what we need to do in the current context as a result. We track the interface state between frames with an enumerated variable, saved as ‘mode‘ with ‘INPUT_MODE‘ datatype – defined as waiting for a new order (‘AWAITING‘), currently making a new selection (‘SELECTING‘), or waiting to confirm an attack order (‘PENDING_ATTACK‘).

At the start of a new selection (the left mouse button having been pressed while we are currently not waiting to confirm an attack order), we let any units we may have previously had selected know they’ve been deselected and re-initialize the selectedUnits array before adding any valid units from this frame’s hoveredUnits to it. In confirming an attack order (the left mouse button having been pressed sometime after the attack button had been pressed), we find any valid enemies within hoveredUnits and order our currently selected units to attack the enemy closest to themselves. If we found no enemies, we issue an attack-move order instead.

  /*...*/
  //Start new selection, or confirm attack order
  if (Input.GetButtonDown("Select")) {  //Has the left mouse button been pressed?
    if (mode == INPUT_MODE.AWAITING) {
      mode = INPUT_MODE.SELECTING;
      UpdateColors(selectingColor);
      if (!Input.GetButton("AddToSelection")) {  //Is the shift button (not) being held down?
        foreach (Unit u in selectedUnits) {
          u.BroadcastMessage("OnUnselect");
        }
        selectedUnits = new Unit[0];
      }

      Unit[] ownedUnits = Unit.UnitsOwnedByArmy(hoveredUnits, playerArmy);
      selectedUnits = Unit.AppendToArray(selectedUnits, ownedUnits);
      foreach (Unit u in ownedUnits) {
        u.BroadcastMessage("OnSelect");
      }
    }
    else if (mode == INPUT_MODE.PENDING_ATTACK) {
      Unit[] enemyUnits = Unit.UnitsEnemyOfArmy(hoveredUnits, playerArmy);
      if (enemyUnits.Length > 0) {
        foreach (Unit u in selectedUnits) {
          u.SendMessage("OnAttackOrder", Unit.ClosestUnit(enemyUnits, u));
        }
      }
      else {
        if (selectedUnits.Length > 1) {
          Unit[] farthestPair = FindFarthestUnits(selectedUnits);
          Vector3 selectedCenter = (farthestPair[0].transform.position + farthestPair[1].transform.position)/2f;
          float selectedRadius = (farthestPair[0].transform.position - farthestPair[1].transform.position).magnitude/2f;
          float selectionRatio = selectionRadius/selectedRadius;
          foreach (Unit u in selectedUnits) {
            Vector3 offset = selectionRatio * (u.transform.position - selectedCenter);
            u.SendMessage("OnAttackMoveOrder", cursorPoint + offset);
          }
        }
        else {
          foreach (Unit u in selectedUnits) {
            u.SendMessage("OnAttackMoveOrder", cursorPoint);
          }
        }
      }
      mode = INPUT_MODE.AWAITING;
      UpdateColors(awaitingColor);
    }
  }
  //Continue selection
  else if (Input.GetButton("Select")) {  //Is the left mouse button being held down?
    Unit[] ownedUnits = Unit.UnitsOwnedByArmy(hoveredUnits, playerArmy);
    selectedUnits = Unit.AppendToArray(selectedUnits, ownedUnits);
    foreach (Unit u in ownedUnits) {
      u.BroadcastMessage("OnSelect");
    }
  }
  //End selection
  else if (Input.GetButtonUp("Select")) {  //Has the left mouse button been released?
    mode = INPUT_MODE.AWAITING;
    UpdateColors(awaitingColor);
  }

  //Move order
  if (Input.GetButtonDown("Order")) {  //Has the right mouse button been pressed?
    if (mode == INPUT_MODE.PENDING_ATTACK) {
      mode = INPUT_MODE.AWAITING;
      UpdateColors(awaitingColor);
    }
    else {
      Unit[] enemyUnits = Unit.UnitsEnemyOfArmy(hoveredUnits, playerArmy);
      if (enemyUnits.Length > 0) {
        foreach (Unit u in selectedUnits) {
          u.SendMessage("OnAttackOrder", Unit.ClosestUnit(enemyUnits, u));
        }
      }
      else {
        if (selectedUnits.Length > 1) {
          Unit[] farthestPair = FindFarthestUnits(selectedUnits);
          Vector3 selectedCenter = (farthestPair[0].transform.position + farthestPair[1].transform.position)/2f;
          float selectedRadius = (farthestPair[0].transform.position - farthestPair[1].transform.position).magnitude/2f;
          float selectionRatio = selectionRadius/selectedRadius;
          foreach (Unit u in selectedUnits) {
            Vector3 offset = selectionRatio * (u.transform.position - selectedCenter);
            u.SendMessage("OnMoveOrder", cursorPoint + offset);
          }
        }
        else {
          foreach (Unit u in selectedUnits) {
            u.SendMessage("OnMoveOrder", cursorPoint);
          }
        }
      }
    }
  }
  /*...*/

Now, take note of the highlighted sections above, in attack-move orders and right-click move orders, because it’s where things get kind of exciting. Each click can indicate an area, which allows room for new mechanisms to emerge. In putting this demo together, I realized we could distribute move orders over the cursor’s area. If we have more than one unit currently selected, we can treat the group itself as a circle with a diameter equal to the distance between the furthest apart pair of units. We can get a center (the midpoint between said farthest pair) and each unit’s vector offset from that center quite simply and easily. Then, taking the ratio of the current selection radius compared to the radius of our selected group, we can scale each unit’s offset from the group’s center to fit within the circular area of our cursor and order each unit individually to move to the scaled-offset position within the cursor’s area, instead of just to the cursor’s center.

In this way, we can have groups of units maintain their formation without any advanced behavioral AI. For this demo implementation, I’m using Unity’s built-in pathfinding to handle unit movement, with various unit speeds, radii, and turning rates. In the order-handling code within the Unit class, I’m simply setting the NavMeshAgent‘s goal location (via SetDestination()) to the target point in space of the order (adjusted to be on the navigation mesh) and nothing else. No desired minimum distances, no crowd behavior code, no representation of formations or collective facing directions, etc. Unity’s pathfinding is already quite robust in and of itself, so the units path around and attempt to push through each other, but don’t know on their own anything about whether or not they’re part of a group of other units or how to stay spread out. Additionally, the player has a new lever of control over how their units group and move, by being able to spread and collapse formations at will with different cursor radii. Units can be told to fan out and cover a wide area or clump up and cluster around a single point. Furthermore, none of this precludes the ability to add the advanced behavior and formation logic of modern RTS’s. There’s room, for example, to calculate which way a group of units is headed or facing, and rotate the formation to match the direction of new orders.

Closing Notes

The implications of circle select aren’t limited to easing box selection’s woes and distributed-area orders, either. Since it relies on only a single raycast and is represented most conveniently as a sphere in world space, it’s open to possibilities for verticality usually foreign to the RTS format. Imagine, for example, a 3D ant colony RTS with a freely-moving camera, where in an underground view you could select groups of ants from within tunnels and order them about.

Now, to be completely fair, circle select isn’t exactly a strictly better interface solution than box selection. Without an easily definable ‘reverse’, you can’t repeal a selected area while selecting – if you drag over a unit you didn’t want to select, you have to subsequently deselect it or start a new selection. While the middle mouse wheel makes perhaps the most sense for the nature of adjusting the radius as an input axis, it comes with the often obnoxiously un-ergonomic and strenuous cost of being the middle mouse wheel (as an aside, the middle mouse button is one of my least favorite, most middle-finger-cramp-inducing aspects of using pretty much all of the 3D modeling software I’ve worked with in my life). And all that’s not to mention the necessary learning curve – box selection isn’t just standard RTS interface, it’s standard PC interface, so it comes as something almost always already understood and pre-taught.

But, all that said, why not give it a try for yourself, and see what you think?