A Controllable Camera

In the previous step we built a 3D scene with a constantly-rotating camera. Wouldn't it be nice if we could move the camera interactively? In this step, we'll build the controls for that.

The Camera Model

For our user-controlled camera, we're going to implement a camera that rotates around a target point. The user will be able to pan the camera to change the target point, tumble the camera to examine the target point from various angles, and dolly the camera closer to or further from the target point. The camera's up direction will always be aligned with the \( \pm z \) axis in order to keep the scene a bit more grounded.

The nice thing about this azimuth-elevation parameterization is that the diagram above shows that we can write down the camera's local right, up, and out directions, as well as its position, in terms of some basic trigonometry on the azimuth (\( \phi \)) and elevation (\( \theta \)) angles:

To understand these, notice that \( c_{\mathrm{out}} \) can be directly read from the picture using the definitions of trigonometric functions. The equation for \( c_{\mathrm{right}} \) uses the fact that rotating \( (x,y) \) 90 degrees counterclockwise is \( (-y, x) \) -- so it's the perpendicular to \( c_{\mathrm{out}} \) in the xy plane. Finally, \( c_{\mathrm{up}} \) is \( c_{\mathrm{out}} \) with the elevation rotated 90 degrees, using the same \( (-y, x) \) idea.

The camera position can be computed by moving by the radius along the out direction from the target point:

With these vectors in hand we can write down the local-from-world transform for the camera (i.e., the camera-from-world transform) as a linear function on homogeneous coordinates:

To see how this function performs a transformation into camera-local space, observe that when applied to a position \( p = \left[ \begin{array}{cccc} x & y & z & 1 \end{array} \right]^\mathsf{T} \), the output, \( p' = M_{\mathrm{camera}\gets\mathrm{world}} p \), has \( p'_\mathrm{x} = (p - c_\mathrm{at}) \cdot c_\mathrm{right} \) as its first coordinate. In other words, \( M_{\mathrm{camera}\gets\mathrm{world}} \) is subtracting the camera position and then projecting onto the camera's local directions. (Remember that we think of cameras as looking along their local \( -z \) axis, which is why we use an "out" direction here.)

The Camera Matrix Code

Since we just wrote down the camera transformation matrix as math, it makes sense to go ahead and implement it into our little math library as code.

Note: if you like being able to immediately test the code you've written, go ahead and start the next section and circle back to this. (Or write some test code in main.cpp!)

We'll start with the function prototype and a skeleton of what we need to do:

//NOTE: I put this code at the bottom of the file, but feel free to organize it differently

//orbit camera matrix:
// makes a camera-from-world matrix for a camera orbiting target_{x,y,z}
//   at distance radius with angles azimuth and elevation.
// azimuth is counterclockwise angle in the xy plane from the x axis
// elevation is angle up from the xy plane
// both are in radians
inline mat4 orbit(
		float target_x, float target_y, float target_z,
		float azimuth, float elevation, float radius
	) {

	//TODO: compute right direction

	//TOOD: compute up direction

	//TODO: compute out direction

	//TODO: compute camera position

	//TODO: assemble and return camera-from-world matrix
}

Now it's time to transcribe the math. We'll start with the right vector. This vector is parallel to the xy plane, so its z component must be zero. And the x and y components can be computed by rotating the azimuth direction 90 degrees counterclockwise:

	//shorthand for some useful trig values:
	float ca = std::cos(azimuth);
	float sa = std::sin(azimuth);
	float ce = std::cos(elevation);
	float se = std::sin(elevation);

	//TODO: compute right direction
	//camera's right direction is azimuth rotated by 90 degrees:
	float right_x =-sa;
	float right_y = ca;
	float right_z = 0.0f;

The up vector is the azimuth direction rotated up by the elevation angle (plus 90 degrees):

	//TODO: compute up direction
	//camera's up direction is elevation rotated 90 degrees:
	// (and points in the same xy direction as azimuth)
	float up_x = -se * ca;
	float up_y = -se * sa;
	float up_z = ce;

The out vector is the azimuth direction rotated up by the elevation angle:

	//TODO: compute out direction
	//direction to the camera from the target:
	float out_x = ce * ca;
	float out_y = ce * sa;
	float out_z = se;

The camera position can be computed by moving away from the target along the out direction:

	//TODO: compute camera position
	//camera's position:
	float eye_x = target_x + radius * out_x;
	float eye_y = target_y + radius * out_y;
	float eye_z = target_z + radius * out_z;

Note that we're using eye as the variable name here for some consistency with the naming in the look_at function.

And the final linear transformation can be assembled by writing tabulating the appropriate dot products and subtractions:

	//TODO: assemble and return camera-from-world matrix
	//camera's position projected onto the various vectors:
	float right_dot_eye = right_x*eye_x + right_y*eye_y + right_z*eye_z;
	float up_dot_eye    =    up_x*eye_x +    up_y*eye_y +    up_z*eye_z;
	float out_dot_eye   =   out_x*eye_x +   out_y*eye_y +   out_z*eye_z;

	//the final local-from-world transformation (column-major):
	return mat4{
		right_x, up_x, out_x, 0.0f,
		right_y, up_y, out_y, 0.0f,
		right_z, up_z, out_z, 0.0f,
		-right_dot_eye, -up_dot_eye, -out_dot_eye, 1.0f,
	};

And that's our azimuth-elevation camera matrix. At this point, the code should compile without warnings or errors. Of course, we're not actually using this code anywhere yet; so let's get into adding free camera support to Tutorial.

Basic Free Camera Support

We've written code that goes from camera parameters to a local-from-world transformation. Let's add support for storing those parameters and selecting the camera to use.

Free Camera Data

First, we need a place to store our free camera's target, azimuth, elevation, and radius. A reasonable location is in the Tutorial struct in the "resources that change when time passes or the user interacts" section.

Note that I'm also going to add variables for the camera's field of view and clipping planes here, just to keep all the ingredients in the final clip-from-world transformation together.

	float time = 0.0f;

	struct OrbitCamera {
		float target_x = 0.0f, target_y = 0.0f, target_z = 0.0f; //where the camera is looking + orbiting
		float radius = 2.0f; //distance from camera to target
		float azimuth = 0.0f; //counterclockwise angle around z axis between x axis and camera direction (radians)
		float elevation = 0.25f * float(M_PI); //angle up from xy plane to camera direction (radians)

		float fov = 60.0f / 180.0f * float(M_PI); //vertical field of view (radians)
		float near = 0.1f; //near clipping plane
		float far = 1000.0f; //far clipping plane
	} free_camera;

	mat4 CLIP_FROM_WORLD;

Also, we'd like to give the user the option to dynamically switch between this free camera and our previous animated camera (let's call it the "scene camera"), so let's add a flag for that and document the behavior.

	float time = 0.0f;

	//for selecting between cameras:
	enum class CameraMode {
		Scene = 0,
		Free = 1,
	} camera_mode = CameraMode::Free;

	//used when camera_mode == CameraMode::Free:
	struct OrbitCamera {
		//...
	} free_camera;

	//computed from the current camera (as set by camera_mode) during update():
	mat4 CLIP_FROM_WORLD;

At this point, the code should compile cleanly, but we still aren't doing anything with these values.

Drawing with the Free Camera

To use our new camera data, all we need to do is update the computation of CLIP_FROM_WORLD at the start of Tutorial::update to select between the old scene camera and the new free camera:

void Tutorial::update(float dt) {
	time = std::fmod(time + dt, 60.0f);

	if (camera_mode == CameraMode::Scene) {
		//camera rotating around the origin:
		float ang = float(M_PI) * 2.0f * 10.0f * (time / 60.0f);
		CLIP_FROM_WORLD = perspective(
			60.0f * float(M_PI) / 180.0f, //vfov
			rtg.swapchain_extent.width / float(rtg.swapchain_extent.height), //aspect
			0.1f, //near
			1000.0f //far
		) * look_at(
			3.0f * std::cos(ang), 3.0f * std::sin(ang), 1.0f, //eye
			0.0f, 0.0f, 0.5f, //target
			0.0f, 0.0f, 1.0f //up
		);
	} else if (camera_mode == CameraMode::Free) {
		CLIP_FROM_WORLD = perspective(
			free_camera.fov,
			rtg.swapchain_extent.width / float(rtg.swapchain_extent.height), //aspect
			free_camera.near,
			free_camera.far
		) * orbit(
			free_camera.target_x, free_camera.target_y, free_camera.target_z,
			free_camera.azimuth, free_camera.elevation, free_camera.radius
		);
	} else {
		assert(0 && "only two camera modes");
	}
	//...
}

The new code to compute the clip-from-world transform has the same structure as the old code, except that we're not using hard-coded values in the perspective matrix computation and we're using the orbit helper we wrote above instead of the look_at helper.

At this point, the code should compile and run, and you should be able to see your scene from the (default, as-initialized) azimuth and elevation values. Now is a good time to exercise your orbit function a bit by changing the default values and re-building / re-running to make sure the camera moves as you expect it to.

Controlling the Camera

Now that we're drawing the scene through our free camera, let's actually let the user move the camera around.

To do this we're going to need to add code to the Tutorial::on_input function to interpret InputEvents. So now is a good time to review the contents of this event structure:

union InputEvent {
	enum Type : uint32_t {
		MouseMotion,
		MouseButtonDown,
		MouseButtonUp,
		MouseWheel,
		KeyDown,
		KeyUp
	} type;
	struct MouseMotion {
		Type type;
		float x, y; //in (possibly fractional) swapchain pixels from upper left of image
		uint8_t state; //all mouse button states, bitfield of (1 << GLFW_MOUSE_BUTTON_*)
	} motion;
	struct MouseButton {
		Type type;
		float x, y; //in (possibly fractional) swapchain pixels from upper left of image
		uint8_t state; //all mouse button states, bitfield of (1 << GLFW_MOUSE_BUTTON_*)
		uint8_t button; //one of the GLFW_MOUSE_BUTTON_* values
		uint8_t mods; //bitfield of modifier keys (GLFW_MOD_* values)
	} button;
	struct MouseWheel {
		Type type;
		float x, y; //scroll offset; +x right, +y up(?); from glfw scroll callback
	} wheel;
	struct KeyInput {
		Type type;
		int key; //GLFW_KEY_* codes
		int mods; //GLFW_MOD_* bits
	} key;
};

In reviewing this structure, there are two important things to notice. First, this is a union -- that is, it is a blob of memory that can be interpreted as any of the contained structure types. This means that -- for example -- evt.type, evt.motion.type, evt.button.type, evt.wheel.type, and evt.key.type are all different names for the same memory location. It also means that InputEvents are a lot smaller than they appear at first glance, since they only need to be large enough to hold their largest member.

Second, according to the comments, the meanings of many of the fields (e.g., evt.button.key depend on GLFW_* constants, so we need to add a header in Tutorial.cpp so our code has these available:

//...
#include "VK.hpp"
#include "refsol.hpp"

#include <GLFW/glfw3.h>

#include <array>
#include <cassert>
//...

It's worth checking that this code compiles. (It should; since other parts of the codebase use the same path to include glfw!)

Toggling Camera Modes

As a warm-up, let's make it possible for the user to toggle camera modes by pressing the tab key.

void Tutorial::on_input(InputEvent const &evt) {
	//general controls:
	if (evt.type == InputEvent::KeyDown && evt.key.key == GLFW_KEY_TAB) {
		//switch camera modes
		camera_mode = CameraMode((int(camera_mode) + 1) % 2);
		return;
	}
}

This code returns after switching modes since we don't want any later event handling code to be allowed to respond to the tab key.

At this point, the code should compile and run and (when you press tab) the view should switch back and forth between the static "free camera" and the rotating "scene camera".

Dollying

We want the user to be able to zoom the camera in and out with the mouse wheel, so let's add that functionality, but make it available only in free camera mode:

void Tutorial::on_input(InputEvent const &evt) {
	//general controls:
	//...

	//free camera controls:
	if (camera_mode == CameraMode::Free) {

		if (evt.type == InputEvent::MouseWheel) {
			//change distance by 10% every scroll click:
			free_camera.radius *= std::exp(std::log(1.1f) * -evt.wheel.y);
			//make sure camera isn't too close or too far from target:
			free_camera.radius = std::max(free_camera.radius, 0.5f * free_camera.near);
			free_camera.radius = std::min(free_camera.radius, 2.0f * free_camera.far);
			return;
		}

	}
}

Since this code acts multiplicatively on the camera's radius, a determined user can scroll into very small (0) or very large (inf) values, which cannot be returned from. The min and max functions place some guard rails to avoid these degenerate cases.

As before, the code returns once it has handled the event.

This code should compile and run, and you should be able to mouse-wheel-roll (or two-finger scroll) to zoom.

Tumbling

Now let's make it so the user can left-click and drag to tumble (rotate) the camera around the scene.

Unlike earlier inputs where our code could respond to a single event, a tumble requires handling a sequence of events: the user starts the tumble by pressing the left mouse button, continues the tumble by moving the mouse, and finishes the tumble by releasing the left mouse button. During the tumble action, no other event handlers should be running (e.g., we don't want the user to be able to switch cameras while tumbling).

To support this, we will use the idea of an "action" -- an event handling function that gets all input until cancelled. I find that action functions like this are a lightweight way to keep all the code relevant to a modal interaction in one place.

	virtual void on_input(InputEvent const &) override;

	//modal action, intercepts inputs:
	std::function< void(InputEvent const &) > action;

	float time = 0.0f;

Now that we have a place to store the action, we can modify our event handling function to check if there is a current action and pass events to it:

void Tutorial::on_input(InputEvent const &evt) {
	//if there is a current action, it gets input priority:
	if (action) {
		action(evt);
		return;
	}
	
	//general controls:
	//...
}

This code should compile and run, but nothing should happen yet (std::function< >s start empty, and thus evaluate to false when tested in a conditional).

Let's start with the skeleton of a tumble action that will be installed on left-mouse-down and will remove itself on left-mouse-up:

void Tutorial::on_input(InputEvent const &evt) {
	//...

	//general controls:
	//...

	//free camera controls:
	if (camera_mode == CameraMode::Free) {

		if (evt.type == InputEvent::MouseWheel) {
			//...
		}

		if (evt.type == InputEvent::MouseButtonDown && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
			//start tumbling

			std::cout << "Tumble started." << std::endl;
			
			action = [this](InputEvent const &evt) {
				if (evt.type == InputEvent::MouseButtonUp && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
					//cancel upon button lifted:
					action = nullptr;

					std::cout << "Tumble ended." << std::endl;
					return;
				}
				if (evt.type == InputEvent::MouseMotion) {
					//TODO: handle motion
					return;
				}
			};

			return;
		}
	}

	//...
}

A brief word about lambdas. The code [this](InputEvent const &evt) { /* ... */ } is a "lambda expression". It defines a local, anonymous function. Like any other C++ function, the part in () is the parameter list and the part in {} is the function body. The important thing to notice here is the code in [], the capture specification. This tells the compiler what variables from the surrounding scope we want to be able to access in the function and how they should be made available (via reference or via copy).

Because this particular function will be used outside of the scope where it was instantiated (i.e., the current invocation of on_input will have returned before action is called), it's important that any local variables are captured by copy and not by reference. The current capture specification, [this], captures a copy of the this pointer, which is as we want.

If you compile and run this code you should be able to press the left mouse button down to get Tumble started. to print and lift it to get Tumble ended. to print.

Okay, now to actually implement the tumbling behavior. To do this, we'll have our code remember the camera parameters and mouse position at the start of the tumble:

if (evt.type == InputEvent::MouseButtonDown && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
	//start tumbling

	std::cout << "Tumble started." << std::endl;

	float init_x = evt.button.x;
	float init_y = evt.button.y;
	OrbitCamera init_camera = free_camera;

	action = [this,init_x,init_y,init_camera](InputEvent const &evt) {
		if (evt.type == InputEvent::MouseButtonUp && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
			//...
			std::cout << "Tumble ended." << std::endl;
			//...
		}
		//...
	};

	return;
}

Notice that -- as per earlier discussion -- I've added the initial values to the action's capture specification by copy and not by reference (to capture by reference, I would have put & in front of each variable name).

Finally, we update the body of the action to actually respond to motions using these initial values. Specifically, it will look at the change in mouse position since the start of the drag and use this to update the azimuth and elevation angles relative to the initial camera position.

action = [this,init_x,init_y,init_camera](InputEvent const &evt) {
	//...
	if (evt.type == InputEvent::MouseMotion) {
		//TODO: handle motion
		//motion, normalized so 1.0 is window height:
		float dx = (evt.motion.x - init_x) / rtg.swapchain_extent.height;
		float dy =-(evt.motion.y - init_y) / rtg.swapchain_extent.height; //note: negated because glfw uses y-down coordinate system

		//rotate camera based on motion:
		float speed = float(M_PI); //how much rotation happens at one full window height
		free_camera.azimuth = init_camera.azimuth - dx * speed;
		free_camera.elevation = init_camera.elevation - dy * speed;

		//reduce azimuth and elevation to [-pi,pi] range:
		const float twopi = 2.0f * float(M_PI);
		free_camera.azimuth -= std::round(free_camera.azimuth / twopi) * twopi;
		free_camera.elevation -= std::round(free_camera.elevation / twopi) * twopi;
		return;
	}
};

This code should compile and run, and you should be able to spin the camera around! Nifty!

One thing that might feel a little weird is that if you start a tumble while the camera is upside-down, the azimuth rotation might seem backward. I like to use the following little adjustment (which I noticed in Blender's camera controls) to compensate for this:

	//rotate camera based on motion:
	float speed = float(M_PI); //how much rotation happens at one full window height
	float flip_x = (std::abs(init_camera.elevation) > 0.5f * float(M_PI) ? -1.0f : 1.0f); //switch azimuth rotation when camera is upside-down
	free_camera.azimuth = init_camera.azimuth - dx * speed * flip_x;
	free_camera.elevation = init_camera.elevation - dy * speed;

With this, you should have a fully functional tumbling, dollying camera. But what if you want to look at some other point? We'll handle panning next.

Panning

Our final missing piece of camera control is panning. When the user shift-left-clicks and drags, they should move the target point of the camera parallel to the camera's image plane.

We can add this action to our handling code above the tumbling action:

//free camera controls:
if (camera_mode == CameraMode::Free) {

	if (evt.type == InputEvent::MouseWheel) {
		//...
	}

	if (evt.type == InputEvent::MouseButtonDown && evt.button.button == GLFW_MOUSE_BUTTON_LEFT && (evt.button.mods & GLFW_MOD_SHIFT)) {
		//start panning
		float init_x = evt.button.x;
		float init_y = evt.button.y;
		OrbitCamera init_camera = free_camera;

		action = [this,init_x,init_y,init_camera](InputEvent const &evt) {
			if (evt.type == InputEvent::MouseButtonUp && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
				//cancel upon button lifted:
				action = nullptr;
				return;
			}
			if (evt.type == InputEvent::MouseMotion) {
				//TODO: handle motion
				return;
			}
		};

		return;
	}

	if (evt.type == InputEvent::MouseButtonDown && evt.button.button == GLFW_MOUSE_BUTTON_LEFT) {
		//start tumbling
		//...
	}
	//...
}

This is a situation where our established pattern of return-ing when an event is handled is important. Without the return statement, execution would fall through and start a tumble action, which would immediately replace the pan action.

We can fill in the body of the pan action by computing the camera's left and up directions and offsetting based on the mouse's distance travelled; as follows:

if (evt.type == InputEvent::MouseMotion) {
	//TODO: handle motion

	//image height at plane of target point:
	float height = 2.0f * std::tan(free_camera.fov * 0.5f) * free_camera.radius;

	//motion, therefore, at target point:
	float dx = (evt.motion.x - init_x) / rtg.swapchain_extent.height * height;
	float dy =-(evt.motion.y - init_y) / rtg.swapchain_extent.height * height; //note: negated because glfw uses y-down coordinate system

	//compute camera transform to extract right (first row) and up (second row):
	mat4 camera_from_world = orbit(
		init_camera.target_x, init_camera.target_y, init_camera.target_z,
		init_camera.azimuth, init_camera.elevation, init_camera.radius
	);

	//move the desired distance:
	free_camera.target_x = init_camera.target_x - dx * camera_from_world[0] - dy * camera_from_world[1];
	free_camera.target_y = init_camera.target_y - dx * camera_from_world[4] - dy * camera_from_world[5];
	free_camera.target_z = init_camera.target_z - dx * camera_from_world[8] - dy * camera_from_world[9];

	return;
}

Note, first, the use of / rtg.swapchain_extent.height * height to convert from pixels to distances at the image plane. This is important because otherwise motion will feel either too fast or too slow, depending on the camera radius.

Notice, also, the use of the orbit function to compute our camera's up- and left- vectors. It would be slightly more efficient to do this at the start of the pan instead of per-motion-event (and to excerpt just the part of the orbit code we actually need). But generally the user's input is at such a low frequency that using a constant amount more floating point per event won't have a noticable impact; and doing things this way keeps the lambdas looking closer to each-other and avoids duplicating the camera math.

Summing it Up

At this point, you have full control of the free camera's transformation. You can tumble, pan, and dolly the camera to get the best angle on any part of your scene.

Going further, you might want to add control for the camera's projection parameters -- the field of view, clipping planes, and (perhaps) even switching between perspective and orthographic projection. You might even consider adding a key that prints out the current camera parameters and a command-line option that can set them -- useful for getting back to a particular view.

If you expand your code to add view-specific techniques (e.g., frustum culling) it can be useful to add yet another camera mode: a "debug camera" that moves like the free camera but for which culling is performed as if the scene is being drawn through, e.g., the free camera. (And, perhaps, in debug camera mode your code can also display a lot of extra information; like bounding boxes and frustums.)