Working with Point Clouds - GCP Workflow

This video complements the GCP Workflow guide to show you a breakdown of Best Practice for Automated Ground Control Points: working with Targets, capturing and processing the Scan, and some Troubleshooting pointers.

 

Download this GCP Workflow article as a PDF.

Contents

Step 1: Prepare your GCP targets

Step 2: Capture your data

Step 3: Save the survey data

Step 4: Configure your georeferencing job

Step 5: Start processing

Step 6: Review Aura’s constellation matching

Step 7: Adjust the point cloud

Step 8: Match targets to the correct landmark

Step 9: Save the constellation file

Step 10: Finish processing

Step 11: View the final output


Introduction

The best way to improve the quality of your scan data is by georeferencing your scans using Emesent’s Ground Control Points (GCPs) – bright white reflective targets that Hovermap can easily see.

GCPs can enable new scanning opportunities and use cases, including long, large, and feature-poor assets such as roads, stadiums, and tunnels, which were previously a challenge for SLAM-based mapping.

The GCP feature can:

  • enhance Hovermap SLAM-based mapping capabilities
  • improve scan accuracy by removing drift
  • position your scan in the real world
  • increase point cloud accuracy
  • detect distortions in your point cloud

The GCP workflow

The GCP workflow is part of the overall scan processing workflow. At the constellation matching stage, Emesent Aura will try to automatically align the survey points. If it can’t do this, you will have the option to do a manual alignment, before resuming processing.

You can skip Process Workflow as it is part of the GCP workflow. Processing the point cloud removes noise, filters-out outliers, and enhances the data quality, which is essential to improve the accuracy of the georeferencing process.

Once you have prepared your GCP targets, the basic process for GCP processing is as follows.

 GCP workflowStep 1: Prepare your GCP targets

Depending on your requirements for the scan, we recommend that you capture several GCPs. Ideally, they will surround your area of interest.

Follow this process for the best results:

  1. Select a target size to suit the capture method: the 250mm-diameter target is better for drone scans, the smaller 150mm-diameter target is better for walking or Backpack scans.
  2. Install at least three targets. The height is not important. The more targets you use, the more accuracy and redundancy you’ll have.
  3. Consider how you distribute the targets. Try to install them at regular intervals around the extent of the project, approx. every 40m, so you don’t accumulate too much drift. In feature-rich environments you can space them further apart (about 50m), or closer together in feature-poor environments (every 10m).
  4. If possible, securely screw or glue your targets in place so that they don’t move during your scan.
  5. When placing your GCPs, orientate them to reflect the scanning beam at less than a 30 degree angle from the perpendicular back to the Hovermap. So for drone scans, the GCPs should be placed horizontally in your capture area; for walking or vehicle scans, the GCPs should be placed perpendicular to your direction of travel.
  6. Before you start scanning, survey-in the center of each target.

Step 2: Capture your data

  1. Perform a continuous scan that captures all the GCPs and the project area.
  2. While scanning, spend a bit more time near each target so that they show up clearly in the point cloud, but stay at least 1.5m away from the target. It’s even better if you can scan near to each target multiple times during the scan (for example, at the start and end of a stope scan) – walk around the target as you point the Hovermap at it.

It’s better to spend more time in the field scanning your project than trying to fix the scan during processing.

 
Step 3: Save the survey data
  1. Save the survey data as a [yourfilename].csv file using the format described below. The format of the CSV file should be as follows: Landmark ID, Longitude (in Eastings), Latitude (in Northings), Height (in metres), and Target Radius (in metres). It is not necessary to include the column headers when loading the CSV file into Aura. However, the number and order of the columns should be maintained as shown in the example below. Ensure that the survey data has exactly 5 columns. as shown below:

S001,500682.435,6961357.168,11.582,0.125
S002,500684.698,6961354.205,11.517,0.125
S003,500689.328,6961351.237,10.998,0.125
S004,500694.064,6961349.341,10.503,0.125
S005,500699.049,6961348.999,9.855,0.125
S006,500703.646,6961350.846,9.414,0.125

The importer will ignore the first line if header strings are detected.

id,x,y,z,radius
S001,500682.435,6961357.168,11.582,0.125
S002,500684.698,6961354.205,11.517,0.125
S003,500689.328,6961351.237,10.998,0.125
S004,500694.064,6961349.341,10.503,0.125
S005,500699.049,6961348.999,9.855,0.125
S006,500703.646,6961350.846,9.414,0.125

The data must be supplied using the right-handed coordinate system (i.e. the elevation 'z' is up).

Creating your survey information in a CSV file using a spreadsheet might look like this:

 ID

 X

 Y

 Z

 Radius

S001

500682.4

6961357

11.582

0.125

S002

500684.7

6961354

11.517

0.125

S003

500689.3

6961351

10.998

0.125

 

The GCP dataset is not part of the point cloud but is associated with it. The structure of the dataset is as follows:

  • There is only one constellation, but each constellation can have multiple landmarks.
  • Each of these landmarks can have multiple targets assigned to them.
For definitions of these concepts, refer to the Emesent Aura Glossary section.
 GCP data set structure
 
Step 4: Configure your georeferencing job
  1. Open Emesent Aura. Make sure you have an active GCP license.
  2. In the Process tab, click Process Scan.
  3. In the Configure New Scan Job panel, select the GCP workflow then click Add GCP Data.

    add GCP data
  4. In the dialog box that displays, click Add Scan Data.
  5. Browse for the folder that contains the unprocessed scan dataset to be georeferenced.       Select that folder.
  6. Emesent Aura will automatically detect any landmarks in your dataset. Landmarks are distinct and recognizable features in the real world that have been accurately measured and identified during a ground survey.
    You can choose to use these identified landmarks or click Import CSV then browse for the .csv file that contains the survey data you would like to use.
Import CSV file

If you have done GCP processing in this directory before, Emesent Aura will remember your previous landmarks. The software will also check to see if there are any missing fields in your survey data.

After you have imported your CSV file, the data is not editable from within the software. If you have made a mistake, modify your CSV file and import it again. The data will be replaced.

 

7.   Click Save.

8.   In the Location field, enter the preferred name for the output folder. Emesent Aura will create this folder, which stores all the processed results and data, as a child directory within the raw scan folder.

change GCP data
9.   Select the processing profile for the Process and GCP workflows. Refer to the Processing Profiles article for more information about which profiles to use and how to create a custom profile.
 

If you are using a custom profile, click Processing Settings then on the GCP tab, make sure Wait for GCP Target Manual Review is toggled on. If not, Emesent Aura will assume that the constellation of detected targets has been successfully matched to the survey points provided. For all in-built profiles, this setting is enabled by default. Refer to the Process Tab section 3.3 of the Aura user manual for more information on processing settings.

10.  Click Save

You can run the processing automatically end-to-end without interaction, although while you’re building familiarity with the process we recommend you visualize and review the target detection and constellation matching results in Emesent’s point cloud viewer for confirmation or adjustment. For more information on processing GCPs, refer to our article Working with Point Clouds - GCP Workflow.

Step 5: Start processing

  1. Click Start to begin processing. The Configure New Scan Job panel is replaced with the Processing Scan panel and shows a progress bar showing how far along you are in your processing job. In addition to the progress bar, the elapsed time of the processing job is shown to the right.
    The directory file path below the progress bar provides a way to identify the dataset source. This is useful if simultaneously processing multiple jobs with the same output folder name. Copying the file path and pasting it on your computer’s file explorer allows you to access the completed files without having to wait for the processing job to be completed.

processing job

Processing can take some time, depending on the size of the dataset and your chosen settings. If you increase the aggressiveness of the processing parameters to find targets that were missed in a previous processing job, the process will take longer.

If a failure occurs during processing, the Retry button becomes available. Click this button to attempt to process the current job from the last successful stage.


Step 6: Review Aura’s constellation matching

During the processing phase, targets are automatically detected in the point cloud. Aura will try to find clusters of points that look like targets and try to align them with the survey points from the uploaded CSV. At roughly halfway through the process, the Status Bar turns blue and will prompt you for manual input. You will now need to check the associations between your landmarks and your targets.

To skip the manual review stage, go to the GCP tab in Processing Settings then disable Wait for GCP Target Manual Review.

  1. Click Load Constellation to open the intermediate point cloud and the associated constellation file.

Because these two files are associated when you remove one file, the other will also close.

Emesent Aura is not designed to handle two separate constellations. If you load more than one constellation, there is a possibility that they will not update properly. It is strongly recommended to load one constellation at a time.

 review GCP

2. If you have existing scans, you will be prompted to remove them. Save any changes you have made to these existing scans then click Remove. This is necessary to prevent issues with global point placement due to differing reference frames. 

remove existing scans
3. Select the intermediate point cloud to display it with an overlay of the targets. This will allow you to see how well the software has detected targets and automatically aligned them to landmarks. They will not necessarily be completely aligned at this stage (this difference is effectively the drift).

The Intensity color scale helps you see identified targets more easily.

4. Go to the GCPs section in the Visualize tab and click on your constellation file to display the Context panel.

    • A green box: Indicates that the target has been associated with a landmark.

    • A blue box: Indicates that the targets haven’t been associated with a landmark. Review these targets and delete them if you find they are not necessary.

The associated landmark is shown with a connecting line.

GCPs visualize

5. If there haven’t been enough targets detected, change the GCP parameters and go through the process again.

6. If there is a visible target that hasn’t been detected, use the Select Points tool to select the undetected target.

7. Go to the Inactive Targets menu on the right, then assign the target (if required) using the Assign tool. The assignment only works with objects in the point cloud that look like a target with high intensity.

Step 7: Adjust the point cloud

Before you make associations between targets and landmarks, it is a good idea to check that the point cloud is in the correct position. From a target identification and processing perspective, they don’t need to be closely aligned. But the advantage of doing this is that it will help you understand which target is which.

Sometimes it can be difficult to see which landmark is associated with its corresponding target in the point cloud. To make it easier to determine the correct associations, it can be helpful to move the point cloud so that the targets line up better with the landmarks in the constellation.

If the position of the point cloud needs adjusting:

  1. Go to the Visualize tab.

  2. In the Point Clouds section, click on your point cloud.

  3. In the Main toolbar, click Translate to move the point cloud.

Step 8: Match targets to the correct landmark

This part of the process allows you to make sure that all target associations are correct. This will influence processing once it resumes. Misaligned targets need to be unassigned and then reassigned correctly.

During alignment, it is recommended to disable Multi-Frame Rendering especially if your hardware is limited or if you are working with large and complex point clouds.

To skip the manual review stage, go to the GCP tab in Processing Settings then disable Wait for GCP Target Manual Review.

To align a target to the correct landmark:

  1. Go to the Visualize tab.

  2. In the GCPs section, select the correct constellation.

  3. In the Context panel, go to Edit Constellation.

In this example, the targets are aligned to the wrong landmarks. The associated landmark is shown with a connecting line. The misaligned targets are highlighted on the right.

 misaligned targets

4. Drag the target (target_3) to the Inactive Targets menu to unassign it. The box will turn blue instead of green.

inactive targets menu

5. Repeat for any other misaligned targets (in this case, target_5).

Both of the target bounding boxes should now be blue. You can delete any inactive targets by selecting the target and then clicking the Trash Target icon.

trash target

6. Reassign targets by dragging them one at a time back to the correct landmark in the Edit Constellation menu. The correct landmark is the one closest to the target.

edit constellation menu

7. After you have realigned both targets, the bounding boxes should become green again.

bounding boxes in green

Step 9: Save the constellation file

Once you are happy with your target associations, go to the Edit Constellation menu and click Save.

save the constellation file

1. Quick save: This option is only available when running the GCP workflow in Emesent Aura. The new file will overwrite the constellation.yaml file that was originally produced by the processing software.
2. Save Constellation: Opens the Save as dialog box.

Step 10: Finish processing

  1. Click Confirm to continue your GCP processing job. This is the point in the GCP processing where the drift in the point cloud will be fixed.

    detecting targets

Clicking Reload Constellation discards your changes and loads the original constellation.

You will get a notification that says the constellation and the associated point cloud have been removed. This is fine. Emesent Aura removes the point cloud and the constellation from the viewport, as it is no longer needed.


Step 11: View the final output

Once processing has finished, you will get a notification that the GCP process is complete.
In the processing panel, you will get a breakdown of all the outputs from your processing job. (This panel is a mirror of the output folder structure).

GCP complete

  1. Click View beside each generated file to load them into the Viewport for analysis or further editing.

    1. Full Point Cloud
      The point cloud with the complete set of data points. The output file type varies depending on the profile that was used in generating the point cloud. The filename includes the output folder name with the output file type appended to it. For example: Output_laz1_4.laz where the output file type is PointCloud LAZ (1.4) and the file is located in the “Output” folder.

    2. Subsampled Point Cloud
      The point cloud containing a subset of points from the original point cloud dataset. This output is only generated if Subsample Point Clouds is enabled in the Output tab of the Processing Settings panel.

    3. Trajectory File
      The data file contains the movement or path of the Hovermap as it acquired the point cloud data.

    4. Ground Control Accuracy Analysis
      A CSV file that provides an assessment of the accuracy achieved during the georeferencing process. It contains information about how well the georeferenced point cloud aligns with the known real-world coordinates of the GCPs.

  2. Click Open Folder to view the output folder.