What is it?
Fundamentally, photogrammetry is about measurement, the measuring of the imaging subject. To perform high-quality photometric measurement, the photographer capturing the photogrammetry data set must follow a rule-based procedure. This procedure will guide the user on how to configure, position, and orient the camera towards the imaging subject in a way that provides the most useful information to the processing software and minimizes the uncertainty in the resulting measurements. These measurements will be as good or as poor as the design of the measurement structure, or lack thereof, that underlies the collection of the photographic data and the means of its subsequent processing.
Recent technological advances in digital cameras, computer processors, and computational techniques, such as sub-pixel image matching, make photogrammetry a portable and powerful technique. It yields extremely dense and precise 3D surface data with a limited number of photos, captured with standard digital photography equipment, in a relatively short period of time. In the last five years, the variety and power of photogrammetry and related processes have increased dramatically.
Video: “Photogrammetry for Rock Art”
Watch this brief video to see an example of a petroglyph rock art panel as a 3D model created using photogrammetry.
How does it work?
CHI uses an image capture technique based on the work of Neffra Matthews and Tommy Noble at the US Bureau of Land Management (BLM). The BLM Tech Note (PDF) and 2010 VAST tutorial, provide additional information. Neffra and Tommy have been improving their photogrammetry methods at the BLM for over 20 years. Their image capture method acquires photo data sets that are software independent and will get the most information-rich results possible from the various photogrammetry software systems on the market.
Currently CHI uses Agisoft PhotoScan Pro software. PhotoScan uses Structure from Motion (SfM) to simultaneously model the camera’s optical system, the camera’s positions and orientations for each photo, and a sparse cloud of points representing matches of the imaging subject’s features as seen by multiple photos. PhotoScan then uses multi-viewpoint stereo algorithms to build a dense point cloud, which can be transformed into a textured 3D model.
Using SfM algorithms, photographic capture sets can be acquired using uncalibrated camera/lens combinations. To generate the information necessary to characterize how light passes from the imaging subject through the given optical system, SfM algorithms need a set of matched point correspondences. These matched points are found in the overlapping photographs of a planned network of images captured from different positions and orientations relative to the imaging subject. How the camera is moved relative to the subject has a great impact on the degree of uncertainty present in the measurements of the associated 3D representation.
The SfM software must take the information contained in the set of photos and optimally solve for three outcomes:
- Model the camera’s interior geometry (calibration) describing how bundles of light rays travel from the imaging subject through the camera’s optics to the digital sensor
- Determine the relative position and orientation of the camera (pose) for each photo relative to the imaging subject
- Generate a sparse point cloud of 3D points from finding and matching locations in two or more photographs that depict the same feature on the imaging subject
In SfM, the camera calibration, pose, and 3D point matches are all solved simultaneously. No separate camera calibration is needed or desired. This feature separates SfM from other photogrammetry algorithms, which require either a pre-calibrated camera or an additional set of photos to calculate a calibration for the camera, before point matching commences. The SfM camera calibration is continually improved throughout the optimization operation, as camera pose and matched point uncertainty residuals are systematically reduced.
When these three operations have yielded a very low uncertainty result, expressed in small fractions of pixels, the role of the SfM algorithm is finished. The uncertainty of the solution is quantified in the form of a Root Mean Squares (RMS) residual, which is equivalent to the statistical concept of a standard deviation. This level of uncertainty will serve to characterize all subsequent measurement operations.
The photogrammetry software must then use Multi-Viewpoint Stereo (MVS) algorithms, informed by the knowledge of camera calibration, pose, and a sparse cloud of low uncertainty 3D points, to build a dense point cloud of a size determined by the user. The size of the dense cloud can reach into the hundreds of millions of points.
The photogrammetry software then employs surfacing algorithms, employing the dense cloud’s 3D point positions and the look angles from the photos to the matched points, to build the geometrical mesh. A texture map is calculated from the pixels of the original photos and the knowledge of how those pixels map onto the 3D geometry. The result is a textured 3D model that can be measured with precision.
Example: Tlingit Helmet – Views of a 3D Photogrammetric Model
The scale for the project is added during the SfM stage of processing. The scale provides the ability to introduce real-world measurement values to the virtual 3D model. At CHI, we accomplish this by adding at least three (and preferably four) calibrated scale bars of known dimension into the scene containing the imaging subject. The scale bars can be on, around, or next to the region of interest. Each scale bar must be included in multiple (at least three) overlapping images. Scale bars are flat, lightweight linear bars in several sizes with printed targets separated by a known, calibrated distance. The software can recognize the targets. The user then enters distances between the targets. Using calibrated scale bars can produce levels of measurement precision well below one tenth of a millimeter.
Measurement structure design is the process of defining a sensor network and the subsequent methods to process the information it collects. In photogrammetry, the sensor network is the camera’s 3D location and orientation for each photo in the capture set in relation to the imaging subject. To get the best results, this network must collect enough data so that the impact of any incorrect data is minimized. This data set must also enable the reduction to a minimum of the 3D measurement uncertainty of the resulting virtual 3D model representing the imaging subject. The design of the measurement structure is influenced by the imaging subject’s 3D features, any restrictions on the placement of cameras and the number of images necessary to satisfy the given “accuracy” and quality requirements. The prerequisite for any successful measurement in any scientific field of data domain is the design of such a measurement structure. Reduction of measurement uncertainty is accomplished through the reduction and elimination of error in photogrammetric image capture and its subsequent virtual 3D reconstruction.
The final resolution of a dense surface model is governed by the image resolution, or ground sample distance (GSD). The resolution of the images is governed by the size of the real-world area represented by each pixel. The GSD is a function of the resolution of the camera sensor, the focal length of the lens, and the distance from the subject.
How to Capture Photos
A crucial element of a successful photogrammetric process is obtaining a “good” photographic sequence. Good photographic sequences are based on a few simple rules. The CHI photogrammetry training class explores the reasons behind these rules and shows how to make informed choices in the face of challenging subjects.
Here are some suggestions for the camera/lens configuration:
- Choose your distance from the subject and focus. Then tape the focus ring in place.
- Use prime lenses rather than zoom lenses. If a zoom lens must be used, use the nearest or farthest extent of the zoom.
- The camera’s aperture must remain constant during the capture sequence. On a 35mm camera, it is good practice not to set the aperture smaller than f/11. With apertures smaller than f/11, diffraction effects occur that blur the image, significantly reducing the camera’s resolution.
- Use the lowest possible ISO setting. The higher the ISO setting, the more electronic noise is generated in the camera sensor. This noise makes the matching of pixels in different photographs more difficult.
- Turn off image stabilization and auto-rotate camera functions.
- The camera should be set to aperture priority mode (preferably f/5.6–f/11 to get the sharpest images).
- To obtain the highest order results, ensure that the camera configuration does not change for a given sequence of photos.
- If a change of camera or lens configuration is necessary, group the subsequent photos together in a different set from the previous photos. Calibrate the sets of photos separately.
How to determine where to take the photographs:
- To maintain a consistent 66% overlap, the camera must be moved a distance equivalent to 34% of the camera’s field of view between photographs, from left to right.
- Be sure to begin the first row of photos positioned such that two-thirds of the field of view is to the left of the imaging subject.
- Ensure the entire subject is covered by at least three frames.
- Proceed systematically from left to right along the length of the subject and take as many photos as necessary to ensure complete coverage.
- For higher quality results and greater imaging redundancy, which helps lower point matching and depth uncertainty:
- Raise the camera vertically and aim the camera downward 15 degrees to re-photograph the previously captured area.
- At the same time, rotate the camera 90 degrees to portrait mode and use the same 66% overlap from left to right.
- When the second row is finished, lower the camera vertically and aim the camera upward 15 degrees to re-photograph the previously captured area.
- Rotate the camera 180 degrees (for a total of 270 degrees), and again capture the area in the same way.
- It is important to maintain a consistent distance from the subject.
- For multi-resolution applications or to increase or decrease resolution, the camera position (closer or farther away from the subject) or change the focal length of the lens (such as 24mm to 50mm) can be change up to a factor of twice or one half the resolution of the previous set of photos.
- Follow this rule for as many sets of photos as necessary to reach the desired resolution.
- Calibrate each set of photos separately.
- Because of the flexibility of this technique, it is possible to obtain high accuracy 3D data from subjects that are at almost any orientation (horizontal, vertical, above, or below) the camera position.
- For round subjects, capture photos every 10 to 15 degrees and overlap the beginning and end photos to complete the circuit.
Archiving the Results
Photogrammetry is archive friendly. Strictly speaking, all of the 3D information required to build a virtual, textured 3D representation is contained in the 2D photos present in a well-designed photogrammetric capture set. Today, the methods of long-term preservation of photographs are well understood. To preserve the textured 3D information of any imaging subject, all that is necessary is to archive the sets of photos and their associated metadata. When a 3D representation is desired, the archived photo sets can be used to generate or re-generate the virtual model. At the current rate of software and computing power development, it is likely that 3D models built from archived photogrammetry image sets will at some point be available “on demand.”
Example: Cuneiform Cone Sequence
The image sequence below shows a 3D model of a section (17mm X 24mm) of a cuneiform cone from the Archaeological Research Collection of the University of Southern California. The sequence is a series of increasing closeups. Each of the images shows the 3D mesh (the underlying geometry) in the upper right, with texture applied in the lower left.