Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
(API) Cam2D_Snapshot depth image returns incorrect depth values
#1
To reproduce: Place a camera in a scene and point it at a 30x50x10 mm block (attached). Take a depth image of it with:
Code:
from robodk import robolink
import numpy as np
import os
import time
RDK = robolink.Robolink()

# Get depth snapshot
cam = RDK.Item('Depth Camera')
cam.setParam('Open')
time.sleep(0.1)
RDK.Cam2D_Snapshot(os.path.join(os.path.abspath('.'), 'tmp.grey32'), cam, 'Depth')
grey32 = np.fromfile('tmp.grey32', dtype='>u4')
w, h = grey32[:2]
img = np.flipud(np.reshape(grey32[2:], (h, w)))
img = (img / np.iinfo(np.uint32).max)  # rescale to float 0.0 to 1.0
FAR_LENGTH = 100  # whatever FAR_LENGTH is set to
img = img * FAR_LENGTH
img = img.astype(np.uint16)

# Save as pointcloud with open3d
import open3d as o3d
FOV = 63.91  # whatever FOV is set to
fy = h / (2*np.tan(np.radians(FOV) / 2))  # FOV is camera field of view
intrinsic = o3d.camera.PinholeCameraIntrinsic(
   width=w,
   height=h,
   fx=fy,  # Use same value for fx and fy
   fy=fy,
   cx=w / 2,
   cy=h / 2,
)

pcd = o3d.geometry.PointCloud.create_from_depth_image(o3d.geometry.Image(img), intrinsic)
o3d.io.write_point_cloud('pcd.xyz', pcd, write_ascii=True)
Open the resulting pcd.xyz file with MeshLab or your pointcloud viewer of choice.

Expected result: the resulting pointcloud is a rectangular block and has dimensions 30x50x10.

Actual result: the resulting pointcloud does not have dimensions 30x50x10 and has angles that are not 90 degrees.

Fixing z-distortion: if I insert the following line just after normalizing img to be between 0 and 1:

Code:
img = img / (2 - img)
then the resulting pointcloud has correct dimensions. Note that the transformation above takes 0 to 0 and 1 to 1 and takes straight lines and flat planes to straight lines and flat planes, but distorts distances and angles. Maybe someone inserted some sort of gamma correction somewhere?


Attached Files
.stl   30x50x10.stl (Size: 684 bytes / Downloads: 58)
#2
Thank you for your detailed feedback. After some internal discussions and testing we were able to fix the depth map with the latest version of RoboDK.
You can download the latest version of RoboDK (version 5.4.3, only Windows binaries have been updated for now):
https://robodk.com/download

It should now be very accurate when you use this formula (32 bit floating-point numerical accuracy):
img = img * FAR_LENGTH - NEAR_LENGTH
Note this is the distance from the sensor plane (the location of the reference frame your camera is attached to).

Let us know if this is confusing or not what you were looking for.
#3
Thanks. I don't see anything like "NEAR_LENGTH" in the docs; do you mean focal length? And it looks like the range of possible values after your suggested translation will be [-NEAR_LENGTH, FAR_LENGTH - NEAR_LENGTH), is that correct?
#4
By NEAR_LENGTH I meant the focal distance. 

The FAR_LENGTH is the focal length plus the working distance.

For example, if you set the focal length to 10 mm and the Working distance is 2000 mm:
  • NEAR_LENGTH = Focal distance = 10 mm
  • FAR_LENGTH = Focal distance + Working distance = 2010 mm
The distance calculation formula should not return negative values. It should give you the distance from the sensor plane (reference frame in RoboDK), instead of the focal point.

Let us know if something doesn't make sense.
#5
For those interested, this example shows how to retrieve the depth map by socket, create a mesh from the point cloud and import it back into RoboDK: https://github.com/RoboDK/RoboDK-API/blo..._Camera.py

Thank you hn-cso for your contribution.
Find useful information about RoboDK and its features by visiting our Online Documentation and by watching tutorials on our YouTube Channel.
#6
Thanks. I have access only to macOS and linux so I haven't had a chance to try out the changes. The version I'm on is 5.4.1.21935 — can I expect the new version (returned by RDK.Version()) to have the form 5.4.3.xxxxx, or will it be date-based? (Your example script refers to a version "5.4.3-2022-06-20".) I want to conditionally rescale depending on the version since I might not be able to update everywhere all at once.
#7
We will publish a new version for Linux and macOS this week.
RDK.Version() will return 5.4.3.22248 or higher.
Find useful information about RoboDK and its features by visiting our Online Documentation and by watching tutorials on our YouTube Channel.
  




Users browsing this thread:
1 Guest(s)