Performance Testing What can you achieve with your MB System?

Performance Testing Note: You can use the Corps of Engineers Standards or set your own. Questions We Might Ask About Our Multibeam Data: How noisy is it? (What is the repeatability?) How much of the swath can I use? How does it compare with single beam? Two Performance Tests in MBMAX Help to Answer Beam Angle Test – Shows multibeam repeatability vs. beam angle. Check Line Statistics – Compares multibeam vs. single beam depths. Compare results to standards or project requirements Bottom TypeHardSoft Maximum Outlier1 ft1 ft Max. Mean Bias<0.1 ft<0.2 ft 95% confidence depth < 15 ft±0.5 ft ±0.5 ft depth 15 – 40 ft±1.0 ft ±1.0 ft depth > 40 ft±1.0 ft±2.0 ft From Table 3-1: Minimum Performance Standards for Corps of Engineers Hydrographic Surveys

Create Your Reference Surface. Location Small area with a very flat, uniform bottom. Slope should be <5% The flat surface is to prevent DGPS position errors from influencing the results. Using lots of overlap and area of nadir depths (in processing) we get the best possible estimation of depth. Preparation and Survey Two sets of 7 parallel lines, crossing at 90 degreesTwo sets of 7 parallel lines, crossing at 90 degrees Separation = water depth. This will allow valid stats out to 75 degrees from nadir. Reference MTX with 1’ x 1’ cells.Reference MTX with 1’ x 1’ cells. Survey at low or high tides to minimize errors. SV cast immediately before survey.Survey at low or high tides to minimize errors. SV cast immediately before survey. Survey the reference lines.Survey the reference lines. Survey the check lines: middle lines NS and EW. Do not use one of the reference lines!Survey the check lines: middle lines NS and EW. Do not use one of the reference lines! Flat, uniform reference area. Sample Survey grid.

Reference Surface: Processing Steps Reference Survey to Surface XYZ 1.Use a User-Defined MTX that encloses your test area. Set the cell size to 30cm x 3cm (metric) or 1’ x 1’ (foot) 2.Load reference survey into MBMAX64. 3.Apply tide and SV corrections. Filter to +/- 45 degree angle limit. Edit out remaining flyers. 4.Save to XYZ (Save One Point Per Cell, Average, 3 Points). User- Defined MTX.

Testing your MB System: Multibeam Check Lines 1.Use the same User-Defined MTX file that you used when creating the reference surface. 2.Load check survey data into MBMAX64. 3.Edit minimally, removing outliers. Do not use angle limits. We want to generate statistics for all available beams. 4.Start the Beam Angle Test from the Tools menu. (Stage 2) 5.Open Reference Surface (Saved XYZ from step 3)

Beam Angle Test Difference Statistics: Reference Minus Check. Comparison Tab is Overall Result X Axis is beam angle. Y Axis is depth difference. x’s show bias. o’s show repeatability. Maximum difference at 95% confidence. Details Tab is Result at Specific Angle Histogram is the distribution. Max Outlier = Max difference. Mean Difference shows bias between reference and the check lines. 95% confidence = 2 * Standard Deviation.

Beam Angle Test in Action a.k.a. “Performance Test”

SINGLE BEAM vs. MULTIBEAM

Check Line Statistics Do single beam and multibeam soundings match up? They better! a.k.a. “Multibeam vs. Single Beam Comparison” Need: 1.Multibeam Reference Surface 2.Single Beam Check Lines (Middle Lines)

Check Line Statistics – Processing Steps Single Beam Check Lines to XYZ 1.Load Single Beam Check Lines (Raw Data) into SBMAX. 2.Save to XYZ Reference Surface (same as beam angle check surface) 3.Use a User-Define MTX. (30cm x 30cm or 1’ x 1’ cells). 4.Load Reference Surface into MBMAX64. 5.Apply tide and SV corrections. Filter to +/- 45 degree angle limit. Edit out remaining flyers. 6.Save to XYZ (Save One Point Per Cell, Average, 3 Points). Check Lines – Running the Test 7.Use the same MTX as step 3. 8.Load Edited SB (All or XYZ formats) into MBMAX64. 9.Start the Check Line Statistics from the Tools menu. 10.Open the Reference Surface XYZ

Results Once again using difference statistics = reference minus check. Histogram shows the distribution and bias. Mean Difference = Bias. This is the main point of the test! Standard Deviation (Sigma): Spread of the difference distribution. 95% Confidence = 2*Sigma. 95% of the differences fall within 2*Sigma. Not bad! Bias less than 0.1 foot! Check Line Statistics

Sample: Single Beam Test a.k.a. “Multibeam vs. Single Beam Comparison”