双目IMU标定kalibr d455 docker ubuntu20

安装RealsenseSDK2

Uninstalling the Packages:Important Removing Debian package is allowed only when no other installed packages directly refer to it. For example removing librealsense2-udev-rules requires librealsense2 to be removed first.Remove a single package with:sudo apt-get purge <package-name>Remove all RealSense™ SDK-related packages with:dpkg -l | grep "realsense" | cut -d " " -f 3 | xargs sudo dpkg --purge

用官方的rs_imu_calibration.py工具进行IMU自校准

sudo apt-get upgradesource ~/.bashrc重启pip install pyrealsense2 -i https://pypi.tuna./simple

输出

kobosp@NUC12U:~/SLAMC/RealSense/librealsense-2.54.1$ python3 ./tools/rs-imu-calibration/rs-imu-calibration.py waiting for realsense device...Device PID: 0B5CDevice name: Intel RealSense D455Serial number: 234322306379Product Line: D400Firmware version: 5.15.0.2Start interactive mode:FOUND ACCEL with fps=100FOUND GYRO with fps=200-------------------------*** Press ESC to Quit ***-------------------------Align to direction: [ 0. -1. 0.] Mounting screw pointing down, device facing outStatus.collect_data[...................]] Direction data collected.Align to direction: [1. 0. 0.] Mounting screw pointing left, device facing outStatus.collect_dataWARNING: MOVING]] Status.collect_dataWARNING: MOVING]] Status.collect_data[...................]] Direction data collected.Align to direction: [0. 1. 0.] Mounting screw pointing up, device facing outStatus.collect_data[...................]] Direction data collected.Align to direction: [-1. 0. 0.] Mounting screw pointing right, device facing outStatus.collect_data[...................]] Direction data collected.Align to direction: [ 0. 0. -1.] Viewing direction facing downStatus.collect_data[...................]] Direction data collected.Align to direction: [0. 0. 1.] Viewing direction facing upStatus.collect_data[...................]] Direction data collected.Would you like to save the raw data? Enter footer for saving files (accel_<footer>.txt and gyro_<footer>.txt)Enter nothing to not save raw data to disk. >Not writing to files.[-0.00014177 -0.00043501 -0.00264089][1000 1000 1000 1000 1000 1000]using 6000 measurements.[[ 1.00871174 0.01403074 0.00818875][ 0.01946245 1.00727115 0.0139031 ][-0.00739261 0.00403233 1.00809991][ 0.08460933 0.14459764 -0.05031226]]residuals: [30.18549263 4.03554285 46.30014224]rank: 4singular: [443.43522606 435.07939681 427.100062 77.42149191]norm (raw data ): 9.731780norm (fixed data): 9.805828 A good calibration will be near 9.806650./tools/rs-imu-calibration/rs-imu-calibration.py:725: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.outfile.write(imu_calib_table.astype('f').tostring())Would you like to write the results to the camera? (Y/N)YWriting calibration to device.Device PID: 0B5CDevice name: Intel RealSense D455Serial number: 234322306379Firmware version: 5.15.0.2SUCCESS: saved calibration to camera.Done.kobosp@NUC12U:~/SLAMC/RealSense/librealsense-2.54.1$ realsense-viewer

只标定imu

kobosp@NUC12U:~/ublox_driver/catkin_ws$ cd /home/kobosp/kalibr/ROS_MakeSpacekobosp@NUC12U:~/kalibr/ROS_MakeSpace$ catkin_make -j4Base path: /home/kobosp/kalibr/ROS_MakeSpaceSource space: /home/kobosp/kalibr/ROS_MakeSpace/srcBuild space: /home/kobosp/kalibr/ROS_MakeSpace/buildDevel space: /home/kobosp/kalibr/ROS_MakeSpace/develInstall space: /home/kobosp/kalibr/ROS_MakeSpace/install######## Running command: "make cmake_check_build_system" in "/home/kobosp/kalibr/ROS_MakeSpace/build"############ Running command: "make -j4" in "/home/kobosp/kalibr/ROS_MakeSpace/build"####[ 10%] Built target cv_utils[ 21%] Built target matIO_test[ 31%] Built target sumpixel_test[ 63%] Built target imu_an[ 73%] Built target polynomial[100%] Built target pnpkobosp@NUC12U:~/kalibr/ROS_MakeSpace$ source ./devel/setup.bash kobosp@NUC12U:~/kalibr/ROS_MakeSpace$ roslaunch imu_utils D455.launch... logging to /home/kobosp/.ros/log/2c069a92-141a-11ee-bb43-29b32abb72a9/roslaunch-NUC12U-15653.logChecking log directory for disk usage. This may take a while.Press Ctrl-C to interruptDone checking log file disk usage. Usage is <1GB.started roslaunch server http://NUC12U:42555/SUMMARY========PARAMETERS* /imu_an/data_save_path: /home/kobosp/kali...* /imu_an/imu_name: D455* /imu_an/imu_topic: /camera/imu* /imu_an/max_cluster: 100* /imu_an/max_time_min: 120* /rosdistro: noetic* /rosversion: 1.16.0NODES/imu_an (imu_utils/imu_an)auto-starting new masterprocess[master]: started with pid [15676]ROS_MASTER_URI=http://localhost:11311setting /run_id to 2c069a92-141a-11ee-bb43-29b32abb72a9process[rosout-1]: started with pid [15701]started core service [/rosout]process[imu_an-2]: started with pid [15708][ INFO] [1687781313.936297119]: Loaded imu_topic: /camera/imu[ INFO] [1687781313.937552707]: Loaded imu_name: D455[ INFO] [1687781313.937835165]: Loaded data_save_path: /home/kobosp/kalibr/ROS_MakeSpace/src/imu_utils-master/data/[ INFO] [1687781313.938207243]: Loaded max_time_min: 120[ INFO] [1687781313.938563194]: Loaded max_cluster: 100gyr x num of Cluster 100gyr y num of Cluster 100gyr z num of Cluster 100acc x num of Cluster 100acc y num of Cluster 100acc z num of Cluster 100wait for imu data.gyr x numData 1427303gyr x start_t 1.68777e+09gyr x end_t 1.68778e+09gyr x dt -------------7201.08 s-------------120.018 min-------------2.0003 hgyr x freq 198.207gyr x period 0.00504524gyr y numData 1427303gyr y start_t 1.68777e+09gyr y end_t 1.68778e+09gyr y dt -------------7201.08 s-------------120.018 min-------------2.0003 hgyr y freq 198.207gyr y period 0.00504524gyr z numData 1427303gyr z start_t 1.68777e+09gyr z end_t 1.68778e+09gyr z dt -------------7201.08 s-------------120.018 min-------------2.0003 hgyr z freq 198.207gyr z period 0.00504524Gyro X C0.1972216.5822 0.164467 0.0356412 0.000213295Bias Instability 1.06511e-05 rad/sBias Instability 8.83496e-06 rad/s, at 469.243 sWhite Noise 4.13847 rad/sWhite Noise 0.00115513 rad/sbias -0.00206807 degree/s-------------------Gyro y C -0.17004130.8382 -1.40131 0.330197 0.000195154Bias Instability 1.14496e-05 rad/sBias Instability 2.27241e-05 rad/s, at 83.2465 sWhite Noise 7.10001 rad/sWhite Noise 0.00197543 rad/sbias -0.0745407 degree/s-------------------Gyro z C 0.894553 22.0909 2.90767 -0.115123 0.00299519Bias Instability 1.93727e-05 rad/sBias Instability 1.59833e-05 rad/s, at 359.63 sWhite Noise 6.31365 rad/sWhite Noise 0.00176709 rad/sbias -0.142922 degree/s-------------------============================================================================================acc x numData 1427303acc x start_t 1.68777e+09acc x end_t 1.68778e+09acc x dt -------------7201.08 s-------------120.018 min-------------2.0003 hacc x freq 198.207acc x period 0.00504524acc y numData 1427303acc y start_t 1.68777e+09acc y end_t 1.68778e+09acc y dt -------------7201.08 s-------------120.018 min-------------2.0003 hacc y freq 198.207acc y period 0.00504524acc z numData 1427303acc z start_t 1.68777e+09acc z end_t 1.68778e+09acc z dt -------------7201.08 s-------------120.018 min-------------2.0003 hacc z freq 198.207acc z period 0.00504524acc X C 2.93071e-05 0.000595005 0.000230195 3.07237e-06 3.21458e-07Bias Instability 0.000391931 m/s^2White Noise 0.0114576 m/s^2-------------------acc y C 5.77612e-06 0.000771649 0.000172597 -1.40639e-06 7.02271e-07Bias Instability 0.000376985 m/s^2White Noise 0.0126382 m/s^2-------------------acc z C 4.47582e-05 0.000674297 0.00020796 2.02422e-05 -2.51004e-07Bias Instability 0.000403212 m/s^2White Noise 0.0131111 m/s^2-------------------[imu_an-2] process has finished cleanlylog file: /home/kobosp/.ros/log/2c069a92-141a-11ee-bb43-29b32abb72a9/imu_an-2*.log^C[rosout-1] killing on exit[master] killing on exitshutting down processing monitor...... shutting down processing monitor completedone

双目IMU标定

Using the Docker Images

Clone and build the docker image

First make sure that you have install docker on your system using the official Docker Get Docker guide. We can then clone and build the docker container using:

git clone /ethz-asl/kalibr.git

cd kalibr

docker build -t kalibr -f Dockerfile_ros1_20_04 . # change this to whatever ubuntu version you want

Mounting a data folder for use in the container

We can now mount the data folder in the container /data path and enter the command prompt. Some more details can be found on the ROS wiki for Docker.

FOLDER=/path/to/your/data/on/host

xhost +local:root

docker run -it -e “DISPLAY” -e “QT_X11_NO_MITSHM=1”

-v “/tmp/.X11-unix:/tmp/.X11-unix:rw”

-v “$FOLDER:/data” kalibr

Inside the docker, running commands

Using the above command you should have entered the docker container bash prompt. From here you should be able to run kalibr on any files that are in your /data directory. You will want to first load your ROS environment variables.

source devel/setup.bash

rosrun kalibr kalibr_calibrate_cameras

–bag /data/cam_april.bag --target /data/april_6x6.yaml

–models pinhole-radtan pinhole-radtan

–topics /cam0/image_raw /cam1/image_raw

docker build -t kalibr -f Dockerfile_ros1_20_04 .xhost +local:rootFOLDER=/media/kobosp/T7/BafFileExt/CalibraStereoImu/data/K522Dsudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrsource devel/setup.bashrosrun kalibr kalibr_calibrate_cameras --models pinhole-equi pinhole-equi --topics /camera/left/image_raw /camera/right/image_raw --bag ../data/SC.bag --target ../data/april6.yamlrosrun kalibr kalibr_calibrate_cameras --models pinhole-radtan pinhole-radtan --topics /camera/left/image_raw /camera/right/image_raw --bag ../data/SC.bag --target ../data/april6.yamlNUC12xhost +local:rootFOLDER=/home/kobosp/kalibr/K522Dsudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrsource devel/setup.bashrosrun kalibr kalibr_calibrate_imu_camera --cam ../data/data/K522D/Stereo-camchain.yaml --target ../data/data/K522D/april6.yaml --imu ../data/data/K522D/ImuPX4.yaml --bag ../data/StereoImuRaw.bag rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bagrosrun kalibr kalibr_calibrate_imu_camera --cam ../data/cam_april-camchain.yaml --target ../data/april_6x6_80x80cm.yaml --imu ../data/imu_adis16448.yaml --bag ../data/imu_april.bagD455xhost +local:rootFOLDER=/home/kobosp/kalibr/K522Dsudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrsource devel/setup.bashrosrun kalibr kalibr_calibrate_cameras --models pinhole-radtan pinhole-radtan --topics /camera/left/image_raw /camera/right/image_raw --bag ../data/StereoCalib.bag --target ../data/april6.yamlrosrun kalibr kalibr_calibrate_cameras --target ../data/april6.yaml --bag ../data/D455_imu_stereo_640.bag --models pinhole-radtan pinhole-radtan pinhole-radtan --topics /camera/infra1/image_rect_raw /camera/infra2/image_rect_raw /camera/color/image_raw --approx-sync 0.01 rosrun kalibr kalibr_calibrate_imu_camera --target ../data/april6.yaml --bag ../data/D455_imu_stereo_640.bag --cam ../data/D455_imu_stereo_640-camchain_delete_color.yaml --imu ../data/imuD455.yaml

D455 result camera only（06260：00-06271：43）

Target configuration====================Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026399999999999996 [m]------------------------------------------------------------------Progress 4833 / 4833 Time remaining: All views have been processed.Starting final outlier filtering...Progress 208 / 208 Time remaining:..................................................................Calibration complete.[ WARN] [1687801238.075189]: Removed 2129 outlier corners.Processed 4833 images with 118 images usedCalibration results ====================Camera-system parameters:cam0 (/camera/infra1/image_rect_raw):type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>distortion: [-0.00688075 -0.00234333 0.00060909 0.00065811] +- [0.00108439 0.00131152 0.00019999 0.00023459]projection: [384.05884023 383.26530635 320.1080508 236.50917899] +- [0.3859936 0.37823843 0.32987974 0.28630809]reprojection error: [-0.000006, -0.000000] +- [0.165921, 0.147421]cam1 (/camera/infra2/image_rect_raw):type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>distortion: [-0.01013082 0.00240729 0.00033559 0.00091218] +- [0.00111506 0.00141249 0.00019435 0.00023051]projection: [384.5611801 383.62000927 321.64557048 236.3452237 ] +- [0.38043891 0.37508606 0.32846757 0.27637926]reprojection error: [0.000003, 0.000000] +- [0.174134, 0.149537]cam2 (/camera/color/image_raw):type: <class 'aslam_cv.libaslam_cv_python.DistortedPinholeCameraGeometry'>distortion: [-0.05045665 0.03465603 -0.0001941 0.00129754] +- [0.00104712 0.00130109 0.00018304 0.00021591]projection: [383.91815154 382.72975225 322.7773466 240.72266608] +- [0.38168558 0.37553962 0.32272415 0.27573455]reprojection error: [0.000003, 0.000000] +- [0.254101, 0.216741]baseline T_1_0:q: [-0.0000719 0.0015757 -0.00010654 0.99999875] +- [0.00061389 0.00084186 0.00008021]t: [-0.09507969 0.00001435 -0.00024005] +- [0.00014507 0.00014493 0.00039113]baseline T_2_1:q: [ 0.00182985 -0.00005559 -0.00128153 0.9999975 ] +- [0.00060247 0.00082951 0.00007543]t: [ 0.03563543 -0.00029688 0.00093371] +- [0.00014078 0.00014335 0.0003784 ]Target configuration====================Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026399999999999996 [m]Results written to:Saving camera chain calibration to file: ../data/D455_imu_stereo_640-camchain.yamlDetailed results written to file: ../data/D455_imu_stereo_640-results-cam.txtGenerating result report...Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_cameras", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_cameras", line 465, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_cameras", line 429, in mainkcc.generateReport(calibrator, reportFile, showOnScreen=not parsed.dontShowReport, graph=G, removedOutlierCorners=removedOutlierCorners);File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_calibration/CameraUtils.py", line 428, in generateReportfig = pl.figure(offset)File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 539, in figurefigManager = new_figure_manager(num, figsize=figsize,File "/usr/lib/python3/dist-packages/matplotlib/backend_bases.py", line 3259, in new_figure_managerreturn cls.new_figure_manager_given_figure(num, fig)File "/usr/lib/python3/dist-packages/matplotlib/backends/_backend_tk.py", line 950, in new_figure_manager_given_figurewindow = tk.Tk(className="matplotlib")File "/usr/lib/python3.8/tkinter/__init__.py", line 2270, in __init__self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)_tkinter.TclError: couldn't connect to display ":0"root@6564102e925a:/catkin_ws# rosrun kalibr kalibr_calibrate_cameras --target ../data/april6.yaml --bag ../data/D455_imu_stereo_640.bag --models pinhole-radtan pinhole-radtan pinhole-radtan --topics /camera/infra1/image_rect_raw /camera/infra2/image_rect_raw /camera/color/image_raw --approx-sync 0.01

imu-stereo calibreate

root@6564102e925a:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --target ../data/april6.yaml --bag ../data/D455_imu_stereo_640.bag --cam ../data/D455_imu_stereo_640-camchain_delete_color.yaml --imu ../data/imuD455.yamlimporting librariesInitializing IMUs:Update rate: 196.0Accelerometer:Noise density: 0.012402312833224254 Noise density (discrete): 0.17363237966513956 Random walk: 0.0003907092658026438Gyroscope:Noise density: 0.0016325510132217242Noise density (discrete): 0.02285571418510414 Random walk: 1.5847437683303787e-05Initializing imu rosbag dataset reader:Dataset:../data/D455_imu_stereo_640.bagTopic: /camera/imuNumber of messages: 20022Reading IMU data (/camera/imu)Read 20022 imu readings over 103.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026399999999999996 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [384.0588402323697, 383.2653063540785]Principal point: [320.1080508015334, 236.50917898753102]Distortion model: radtanDistortion coefficients: [-0.006880752786807848, -0.0023433306075344223, 0.0006090853603049621, 0.0006581097108187326]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [384.5611801039632, 383.6200092655924]Principal point: [321.6455704836215, 236.34522369515722]Distortion model: radtanDistortion coefficients: [-0.010130821210079678, 0.0024072920780815617, 0.0003355938942277863, 0.0009121809185809588]baseline: [[ 0.99999501 -0.00021331 -0.00315139 -0.09507969][ 0.00021285 0.99999997 -0.00014413 0.00001435][ 0.00315142 0.00014346 0.99999502 -0.00024005][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/D455_imu_stereo_640.bagTopic: /camera/infra1/image_rect_rawNumber of images: 6144Extracting calibration target cornersExtracted corners for 4727 images (of 6144 images)Initializing camera rosbag dataset reader:Dataset:../data/D455_imu_stereo_640.bagTopic: /camera/infra2/image_rect_rawNumber of images: 6144Extracting calibration target cornersExtracted corners for 4740 images (of 6144 images)Baseline between cam0 and cam1 set to:T= [[ 0.99999501 -0.00021331 -0.00315139 -0.09507969][ 0.00021285 0.99999997 -0.00014413 0.00001435][ 0.00315142 0.00014346 0.99999502 -0.00024005][ 0.0.0.1. ]]Baseline: 0.09507999376896113 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 10309 knots (100.000000 knots per second over 103.088201 seconds)Time shift camera to imu (t_imu = t_cam + shift):0.0Estimating time shift camera to imu:Initializing a pose spline with 10309 knots (100.000000 knots per second over 103.088201 seconds)Time shift camera to imu (t_imu = t_cam + shift):0.0Estimating imu-camera rotation priorInitializing a pose spline with 10309 knots (100.000000 knots per second over 103.088201 seconds)Gravity was intialized to [ 0.14048636 -9.79874244 -0.365148 ] [m/s^2]Orientation prior camera-imu found as: (T_i_c)[[ 0.99998713 0.0022974 -0.00452438][-0.00229432 0.99999713 0.00068599][ 0.00452594 -0.0006756 0.99998953]]Gyro bias prior found as: (b_gyro)[ 0.0005352 0.00028102 -0.00355443]Initializing a pose spline with 10321 knots (100.000000 knots per second over 103.208201 seconds)Initializing the bias splines with 5160 knotsAdding camera error terms (/camera/infra1/image_rect_raw)Added 4727 camera error terms Adding camera error terms (/camera/infra2/image_rect_raw)Added 4740 camera error terms Adding accelerometer error terms (/camera/imu)Added 20022 of 20022 accelerometer error terms (skipped 0 out-of-bounds measurements)Adding gyroscope error terms (/camera/imu)Added 20022 of 20022 gyroscope error terms (skipped 0 out-of-bounds measurements)Before Optimization===================Normalized Residuals----------------------------Reprojection error (cam0):mean 1.5040180051522591, median 1.1644218973015361, std: 1.2723949792765081Reprojection error (cam1):mean 1.538573393809447, median 1.1719221799177355, std: 2.476639609899708Gyroscope error (imu0): mean 11.339976914241392, median 7.5091688474891045, std: 13.379757305005842Accelerometer error (imu0): mean 8.153486038455958, median 5.242075257957414, std: 9.121646519762265Residuals----------------------------Reprojection error (cam0) [px]:mean 1.5040180051522591, median 1.1644218973015361, std: 1.2723949792765081Reprojection error (cam1) [px]:mean 1.538573393809447, median 1.1719221799177355, std: 2.476639609899708Gyroscope error (imu0) [rad/s]:mean 0.25918327121758045, median 0.17162741694589884, std: 0.3058039088292728Accelerometer error (imu0) [m/s^2]: mean 1.4157091834235993, median 0.9101940014228962, std: 1.5838131916905602Optimizing...Using the block_cholesky linear system solverUsing the levenberg_marquardt trust region policyUsing the block_cholesky linear system solverUsing the levenberg_marquardt trust region policyInitializingOptimization problem initialized with 20661 design variables and 1214189 error termsThe Jacobian matrix is 2468420 x 92956[0.0]: J: 1.64373e+07[1]: J: 544920, dJ: 1.58924e+07, deltaX: 0.730296, LM - lambda:10 mu:2[2]: J: 81664.1, dJ: 463256, deltaX: 0.233955, LM - lambda:3.33333 mu:2[3]: J: 69063.7, dJ: 12600.3, deltaX: 0.0995945, LM - lambda:1.11111 mu:2[4]: J: 69001.5, dJ: 62.2247, deltaX: 0.156116, LM - lambda:0.37037 mu:2[5]: J: 69001.5, dJ: 0.0283328, deltaX: 0.0819353, LM - lambda:0.123457 mu:2[6]: J: 69001.5, dJ: 0.000615787, deltaX: 0.381697, LM - lambda:0.0411523 mu:2After Optimization (Results)==================Normalized Residuals----------------------------Reprojection error (cam0):mean 0.1758896130775895, median 0.14245583696430716, std: 0.13371268954518678Reprojection error (cam1):mean 0.17891016510212035, median 0.14294664177228214, std: 0.1386604801226171Gyroscope error (imu0): mean 0.38670110529162394, median 0.33661336865893376, std: 0.2409551129101659Accelerometer error (imu0): mean 0.43657983575907644, median 0.3707133012605685, std: 0.3372867104396925Residuals----------------------------Reprojection error (cam0) [px]:mean 0.1758896130775895, median 0.14245583696430716, std: 0.13371268954518678Reprojection error (cam1) [px]:mean 0.17891016510212035, median 0.14294664177228214, std: 0.1386604801226171Gyroscope error (imu0) [rad/s]:mean 0.008838329937609218, median 0.007693538944953683, std: 0.00550792114249Accelerometer error (imu0) [m/s^2]: mean 0.07580439579666423, median 0.06436783267139229, std: 0.05856389416307067Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [[ 0.99997614 0.00107025 -0.00682475 0.01904526][-0.00103482 0.99998598 0.00519263 -0.01114796][ 0.00683022 -0.00518544 0.99996323 -0.03624201][ 0.0.0.1. ]]cam0 to imu0 time: [s] (t_imu = t_cam + shift)-0.0017068548726441536 Transformation T_cam1_imu0 (imu0 to cam1, T_ci): [[ 0.99994985 0.00087329 -0.0099771 -0.07591793][-0.00082296 0.99998692 0.00504706 -0.01112432][ 0.00998138 -0.00503859 0.99993749 -0.03642346][ 0.0.0.1. ]]cam1 to imu0 time: [s] (t_imu = t_cam + shift)-0.0017139831600678141 IMU0:----------------------------Model: calibratedUpdate rate: 196.0Accelerometer:Noise density: 0.012402312833224254 Noise density (discrete): 0.17363237966513956 Random walk: 0.0003907092658026438Gyroscope:Noise density: 0.0016325510132217242Noise density (discrete): 0.02285571418510414 Random walk: 1.5847437683303787e-05T_ib (imu0 to imu0)[[1. 0. 0. 0.][0. 1. 0. 0.][0. 0. 1. 0.][0. 0. 0. 1.]]time offset with respect to IMU0: 0.0 [s]Results written to:Saving camera chain calibration to file: ../data/D455_imu_stereo_640-camchain-imucam.yamlSaving imu calibration to file: ../data/D455_imu_stereo_640-imu.yamlDetailed results written to file: ../data/D455_imu_stereo_640-results-imucam.txtGenerating result report...Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 235, in mainutil.generateReport(iCal, filename=reportFile, showOnScreen=not parsed.dontShowReport)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccUtil.py", line 164, in generateReportfig = pl.figure(offset)File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 539, in figurefigManager = new_figure_manager(num, figsize=figsize,File "/usr/lib/python3/dist-packages/matplotlib/backend_bases.py", line 3259, in new_figure_managerreturn cls.new_figure_manager_given_figure(num, fig)File "/usr/lib/python3/dist-packages/matplotlib/backends/_backend_tk.py", line 950, in new_figure_manager_given_figurewindow = tk.Tk(className="matplotlib")File "/usr/lib/python3.8/tkinter/__init__.py", line 2270, in __init__self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)_tkinter.TclError: couldn't connect to display ":0"root@6564102e925a:/catkin_ws#

kobosp@NUC8i5:~/ETHkalibr$ xhost +local:rootnon-network local connections being added to access control listkobosp@NUC8i5:~/ETHkalibr$ FOLDER=/home/kobosp/ETHkalibr/data/K522D/kobosp@NUC8i5:~/ETHkalibr$ sudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibr[sudo] kobosp 的密码： 102对不起，请重试。[sudo] kobosp 的密码： root@9dae1e48b945:/catkin_ws# exitexitkobosp@NUC8i5:~/ETHkalibr$ sudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrroot@daec1d724e26:/catkin_ws# source devel/setup.bashroot@daec1d724e26:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/RestampStereoImu.bagimporting librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/RestampStereoImu.bagTopic: /mavros/imu/data_rawTraceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 154, in mainimus.append( sens.IccImu(imuConfig, parsed, isReferenceImu=(not imus), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 593, in __init__self.dataset = initImuBagDataset(parsed.bagfile[0], imuConfig.getRosTopic(), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 34, in initImuBagDatasetreader = kc.BagImuDatasetReader(bagfile, topic, bag_from_to=from_to, \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ImuDatasetReader.py", line 31, in __init__self.bag = rosbag.Bag(bagfile)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 467, in __init__self._open(f, mode, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1428, in _openif mode == 'r': self._open_read(f, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1450, in _open_readself._file= open(f, 'rb')FileNotFoundError: [Errno 2] No such file or directory: '../data/RestampStereoImu.bag'root@daec1d724e26:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/RestampStereoImu.bagexitimporting librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/RestampStereoImu.bagexitTopic: /mavros/imu/data_rawTraceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 154, in mainimus.append( sens.IccImu(imuConfig, parsed, isReferenceImu=(not imus), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 593, in __init__self.dataset = initImuBagDataset(parsed.bagfile[0], imuConfig.getRosTopic(), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 34, in initImuBagDatasetreader = kc.BagImuDatasetReader(bagfile, topic, bag_from_to=from_to, \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ImuDatasetReader.py", line 31, in __init__self.bag = rosbag.Bag(bagfile)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 467, in __init__self._open(f, mode, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1428, in _openif mode == 'r': self._open_read(f, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1450, in _open_readself._file= open(f, 'rb')FileNotFoundError: [Errno 2] No such file or directory: '../data/RestampStereoImu.bagexit'root@daec1d724e26:/catkin_ws# exitexitkobosp@NUC8i5:~/ETHkalibr$ FOLDER=/home/kobosp/ETHkalibr/data/K522D/kobosp@NUC8i5:~/ETHkalibr$ sudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrsource devel/setup.bashroot@026e7d150c80:/catkin_ws# source devel/setup.bashroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/RestampStereoImu.bagimporting librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/RestampStereoImu.bagTopic: /mavros/imu/data_rawTraceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 154, in mainimus.append( sens.IccImu(imuConfig, parsed, isReferenceImu=(not imus), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 593, in __init__self.dataset = initImuBagDataset(parsed.bagfile[0], imuConfig.getRosTopic(), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 34, in initImuBagDatasetreader = kc.BagImuDatasetReader(bagfile, topic, bag_from_to=from_to, \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ImuDatasetReader.py", line 31, in __init__self.bag = rosbag.Bag(bagfile)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 467, in __init__self._open(f, mode, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1428, in _openif mode == 'r': self._open_read(f, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1450, in _open_readself._file= open(f, 'rb')FileNotFoundError: [Errno 2] No such file or directory: '../data/RestampStereoImu.bag'root@026e7d150c80:/catkin_ws# lsbuild devel logs srcroot@026e7d150c80:/catkin_ws# cd ..root@026e7d150c80:/# lsbin data home lib64 mnt rootsbin tmpboot dev lib libx32 opt ros_entrypoint.sh srv usrcatkin_ws etc lib32 media proc runsys varroot@026e7d150c80:/# cd data/root@026e7d150c80:/data# lsImuPX4.yaml Stereo-results-cam.txt StereoImuRestamp.bagStereo-camchain.yaml Stereo.bag april6.yamlStereo-report-cam.pdf StereoImu.bag restamp_bag.pyroot@026e7d150c80:/data# cd ../catkin_ws/root@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImu../data/StereoImu.bag ../data/StereoImuRestamp.bagroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610camera/imu 1599 msgs : sensor_msgs/Imu /color 32 msgExtracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag --timeoffset-padding 0.1importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.100000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag --timeoffset-padding 0.005importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.005000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@026e7d150c80:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --himporting librariesCalibrate the spatial and temporal parameters of an IMU to a camera chain.usage: Example usage to calibrate a camera system against an IMU using an aprilgrid.Temporal calibration is enabled by default.kalibr_calibrate_imu_camera --bag MYROSBAG.bag --cam camchain.yaml --imu imu.yaml \--target aprilgrid.yaml camchain.yaml: is the camera-system calibration output of the multiple-cameracalibratin tool (kalibr_calibrate_cameras)example aprilgrid.yaml: | example imu.yaml: (ADIS16448)target_type: 'aprilgrid' |accelerometer_noise_density: 0.006 tagCols: 6|accelerometer_random_walk: 0.0002tagRows: 6|gyroscope_noise_density: 0.0004tagSize: 0.088 |gyroscope_random_walk: 4.0e-06tagSpacing: 0.3 |update_rate: 200.0optional arguments:-h, --help show this help message and exitDataset source:--bag BAGFILE Ros bag file containing image and imu data (rostopicsspecified in the yamls)--bag-from-to bag_from_to bag_from_toUse the bag data starting from up to this time [s]--bag-freq bag_freq Frequency to extract features at [hz]--perform-synchronizationPerform a clock synchronization according to 'Clocksynchronization algorithms for network measurements'by Zhang et al. (2002).Camera system configuration:--cams CHAIN_YAMLCamera system configuration as yaml file--recompute-camera-chain-extrinsicsRecompute the camera chain extrinsics. This option isexclusively recommended for debugging purposes inorder to identify problems with the camera chainextrinsics.--reprojection-sigma REPROJECTION_SIGMAStandard deviation of the distribution of reprojectedcorner points [px]. (default: 1.0)IMU configuration:--imu IMU_YAMLS [IMU_YAMLS ...]Yaml files holding the IMU noise parameters. The firstIMU will be the reference IMU.--imu-delay-by-correlationEstimate the delay between multiple IMUs bycorrelation. By default, no temporal calibrationbetween IMUs will be performed.--imu-models IMU_MODELS [IMU_MODELS ...]The IMU models to estimate. Currently supported are'calibrated', 'scale-misalignment' and 'scale-misalignment-size-effect'.Calibration target:--target TARGET_YAML Calibration target configuration as yaml fileOptimization options:--no-time-calibrationDisable the temporal calibration--max-iter MAX_ITER Max. iterations (default: 30)--recover-covariance Recover the covariance of the design variables.--timeoffset-padding TIMEOFFSET_PADDINGMaximum range in which the timeoffset may changeduring estimation [s] (default: 0.03)Output options:--show-extractionShow the calibration target extraction. (disablesplots)--extraction-steppingShow each image during calibration target extraction(disables plots)--verbose Verbose output (disables plots)--dont-show-report Do not show the report on screen after calibration.--export-poses Also export the optimized poses.root@026e7d150c80:/catkin_ws# exitexitkobosp@NUC8i5:~/ETHkalibr$ FOLDER=/home/kobosp/ETHkalibr/data/Sample/kobosp@NUC8i5:~/ETHkalibr$ sudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibr[sudo] kobosp 的密码： root@dd9c9d890ee0:/catkin_ws# source devel/setup.bashroot@dd9c9d890ee0:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/cam_april-camchain.yaml --target ../data/april_6x6_80x80cm.yaml --imu ../data/imu_adis16448.yaml --bag ../data/imu_april.bagimporting librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.01 Noise density (discrete): 0.1414213562373095 Random walk: 0.0002Gyroscope:Noise density: 0.005Noise density (discrete): 0.07071067811865475 Random walk: 4e-06Initializing imu rosbag dataset reader:Dataset:../data/imu_april.bagTopic: /imu0Number of messages: 14381Reading IMU data (/imu0)Read 14381 imu readings over 71.9 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026399999999999996 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [458.9432286546919, 457.5637533402653]Principal point: [367.0272509347057, 249.3128033381081]Distortion model: radtanDistortion coefficients: [-0.2879529995338575, 0.0781311194952221, 0.00021265916642721963, -0.0001221450347654466]baseline: no data availableInitializing camera rosbag dataset reader:Dataset:../data/imu_april.bagTopic: /cam0/image_rawNumber of images: 1439Extracting calibration target cornersExtracted corners for 1399 images (of 1439 images)Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 7190 knots (100.000000 knots per second over 71.900000 seconds)Time shift camera to imu (t_imu = t_cam + shift):0.0Estimating imu-camera rotation priorInitializing a pose spline with 7190 knots (100.000000 knots per second over 71.900000 seconds)Gravity was intialized to [-0.07968517 -9.56930933 -2.1425] [m/s^2]Orientation prior camera-imu found as: (T_i_c)[[ 0.01540089 0.99947646 -0.02845369][-0.999860.01558051 0.00610171][ 0.00654184 0.02835574 0.99957649]]Gyro bias prior found as: (b_gyro)[-0.00315449 0.02495663 0.07872229]Initializing a pose spline with 7202 knots (100.000000 knots per second over 72.020000 seconds)Initializing the bias splines with 3601 knotsAdding camera error terms (/cam0/image_raw)Added 1399 camera error terms Adding accelerometer error terms (/imu0)Added 14381 of 14381 accelerometer error terms (skipped 0 out-of-bounds measurements)Adding gyroscope error terms (/imu0)Added 14381 of 14381 gyroscope error terms (skipped 0 out-of-bounds measurements)Before Optimization===================Normalized Residuals----------------------------Reprojection error (cam0):mean 1.4750965187964908, median 1.1164274136943468, std: 1.275962492544892Gyroscope error (imu0): mean 1.5735893087893817, median 1.07738778908564, std: 2.0302734131968565Accelerometer error (imu0): mean 5.596232978580502, median 3.9880924156602364, std: 5.727625438992933Residuals----------------------------Reprojection error (cam0) [px]:mean 1.4750965187964908, median 1.1164274136943468, std: 1.275962492544892Gyroscope error (imu0) [rad/s]:mean 0.1112695671047624, median 0.07618282116300379, std: 0.1435681342546Accelerometer error (imu0) [m/s^2]: mean 0.7914268576508128, median 0.5640014382223985, std: 0.8100085576016959Optimizing...Using the block_cholesky linear system solverUsing the levenberg_marquardt trust region policyUsing the block_cholesky linear system solverUsing the levenberg_marquardt trust region policyInitializingOptimization problem initialized with 14423 design variables and 203069 error termsThe Jacobian matrix is 434898 x 64887[0.0]: J: 1.6801e+06[1]: J: 55315.6, dJ: 1.62479e+06, deltaX: 0.534876, LM - lambda:10 mu:2[2]: J: 44083.6, dJ: 11232, deltaX: 0.130086, LM - lambda:3.33333 mu:2[3]: J: 43975.7, dJ: 107.897, deltaX: 0.0919146, LM - lambda:1.11111 mu:2[4]: J: 43975.1, dJ: 0.572779, deltaX: 0.0324899, LM - lambda:0.37037 mu:2Last step was a regression. Reverting[5]: J: 43975.1, dJ: -0.000337646, deltaX: 0.0675993, LM - lambda:0.123457 mu:2After Optimization (Results)==================Normalized Residuals----------------------------Reprojection error (cam0):mean 0.4068389856313736, median 0.35019152049973784, std: 0.2681391818575475Gyroscope error (imu0): mean 0.11010219799507329, median 0.0949146151684012, std: 0.06756248689227881Accelerometer error (imu0): mean 0.3517593093338472, median 0.31929933142341543, std: 0.19736850118762259Residuals----------------------------Reprojection error (cam0) [px]:mean 0.4068389856313736, median 0.35019152049973784, std: 0.2681391818575475Gyroscope error (imu0) [rad/s]:mean 0.007785401082586021, median 0.006711476801928803, std: 0.004777389263535758Accelerometer error (imu0) [m/s^2]: mean 0.04974627859509194, median 0.04515574449556558, std: 0.027912121116478616Transformation T_cam0_imu0 (imu0 to cam0, T_ci): [[ 0.01477499 0.99954017 -0.02647936 0.06553522][-0.99988927 0.01481672 0.00138036 -0.0199015 ][ 0.00177206 0.02645603 0.99964841 -0.00491211][ 0.0.0.1. ]]cam0 to imu0 time: [s] (t_imu = t_cam + shift)-5.358274956430555e-05 IMU0:----------------------------Model: calibratedUpdate rate: 200.0Accelerometer:Noise density: 0.01 Noise density (discrete): 0.1414213562373095 Random walk: 0.0002Gyroscope:Noise density: 0.005Noise density (discrete): 0.07071067811865475 Random walk: 4e-06T_i_b[[1. 0. 0. 0.][0. 1. 0. 0.][0. 0. 1. 0.][0. 0. 0. 1.]]time offset with respect to IMU0: 0.0 [s]Saving camera chain calibration to file: ../data/imu_april-camchain-imucam.yamlSaving imu calibration to file: ../data/imu_april-imu.yamlDetailed results written to file: ../data/imu_april-results-imucam.txtGenerating result report.../catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccPlots.py:187: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.ax=pl.subplot(3, 1, r+1)/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccPlots.py:106: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.pl.subplot(3, 1, i+1)/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccPlots.py:156: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.ax=pl.subplot(3, 1, r+1)/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccPlots.py:126: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.pl.subplot(3, 1, i+1)(kalibr_calibrate_imu_camera:32): dbind-WARNING **: 05:50:43.099: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-Y82OJPuqKt: Connection refused(kalibr_calibrate_imu_camera:32): Gtk-WARNING **: 05:50:43.386: Error loading theme icon 'dialog-error' for stock: Icon 'dialog-error' not present in theme Yaru(kalibr_calibrate_imu_camera:32): Gtk-WARNING **: 05:50:43.391: Error loading theme icon 'dialog-error' for stock: Icon 'dialog-error' not present in theme Yaru(kalibr_calibrate_imu_camera:32): Gtk-WARNING **: 05:50:43.392: Error loading theme icon 'dialog-warning' for stock: Icon 'dialog-warning' not present in theme Yaru(kalibr_calibrate_imu_camera:32): Gtk-WARNING **: 05:50:43.392: Error loading theme icon 'dialog-information' for stock: Icon 'dialog-information' not present in theme YaruReport written to ../data/imu_april-report-imucam.pdfroot@dd9c9d890ee0:/catkin_ws# exitexitkobosp@NUC8i5:~/ETHkalibr$ FOLDER=/home/kobosp/ETHkalibr/data/K522D/kobosp@NUC8i5:~/ETHkalibr$ sudo docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" -v "$FOLDER:/data" kalibrroot@e9491cc5700e:/catkin_ws# source devel/setup.bashroot@e9491cc5700e:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/RestampStereoImu.bagimporting librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/RestampStereoImu.bagTopic: /mavros/imu/data_rawTraceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 154, in mainimus.append( sens.IccImu(imuConfig, parsed, isReferenceImu=(not imus), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 593, in __init__self.dataset = initImuBagDataset(parsed.bagfile[0], imuConfig.getRosTopic(), \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 34, in initImuBagDatasetreader = kc.BagImuDatasetReader(bagfile, topic, bag_from_to=from_to, \File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ImuDatasetReader.py", line 31, in __init__self.bag = rosbag.Bag(bagfile)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 467, in __init__self._open(f, mode, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1428, in _openif mode == 'r': self._open_read(f, allow_unindexed)File "/opt/ros/noetic/lib/python3/dist-packages/rosbag/bag.py", line 1450, in _open_readself._file= open(f, 'rb')FileNotFoundError: [Errno 2] No such file or directory: '../data/RestampStereoImu.bag'root@e9491cc5700e:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImuRestamp.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImuRestamp.bagTopic: /mavros/imu/data_rawNumber of messages: 1Reading IMU data (/mavros/imu/data_raw)Progress 1 / 1 Time remaining: [FATAL] [1659506386.591236]: Could not find any IMU messages. Please check the dataset.root@e9491cc5700e:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImu.bag importing librariesInitializing IMUs:Update rate: 150.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.16139929653597607 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.009261395848684701 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026487999999999998 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: equidistantDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: equidistantDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@e9491cc5700e:/catkin_ws# rosrun kalibr kalibr_calibrate_imu_camera --cam ../data/Stereo-camchain.yaml --target ../data/april6.yaml --imu ../data/ImuPX4.yaml --bag ../data/StereoImu.bag importing librariesInitializing IMUs:Update rate: 200.0Accelerometer:Noise density: 0.013178197378576464 Noise density (discrete): 0.18636785460412406 Random walk: 0.0003957063244250979Gyroscope:Noise density: 0.0007561898045069295Noise density (discrete): 0.01069413877261959 Random walk: 3.64801041874673e-05Initializing imu rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /mavros/imu/data_rawNumber of messages: 63496Reading IMU data (/mavros/imu/data_raw)Read 63496 imu readings over 322.1 secondsInitializing calibration target:Type: aprilgridTags: Rows: 6Cols: 6Size: 0.088 [m]Spacing 0.026399999999999996 [m]Initializing camera chain:Camera chain - cam0:Camera model: pinholeFocal length: [773.03583, 777.516443]Principal point: [672.617788, 372.522376]Distortion model: radtanDistortion coefficients: [-0.297926, 0.077062, 3e-05, -0.000558]baseline: no data availableCamera chain - cam1:Camera model: pinholeFocal length: [773.934111, 778.673235]Principal point: [599.921196, 373.44009]Distortion model: radtanDistortion coefficients: [-0.303576, 0.082196, 0.000697, -0.001685]baseline: [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Initializing camera rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /camera/left/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1574 images (of 1610 images)Initializing camera rosbag dataset reader:Dataset:../data/StereoImu.bagTopic: /camera/right/image_rawNumber of images: 1610Extracting calibration target cornersExtracted corners for 1581 images (of 1610 images)Baseline between cam0 and cam1 set to:T= [[ 0.99997573 0.0011333 -0.00687465 0.07017293][-0.00120622 0.99994295 -0.01061327 -0.00005874][ 0.00686223 0.01062131 0.9999 -0.0001976 ][ 0.0.0.1. ]]Baseline: 0.07017323260714177 [m]Building the problemSpline order: 6Pose knots per second: 100Do pose motion regularization: Falsexddot translation variance: 1000000.000000xddot rotation variance: 100000.000000Bias knots per second: 50Do bias motion regularization: TrueBlake-Zisserman on reprojection errors -1Acceleration Huber width (sigma): -1.000000Gyroscope Huber width (sigma): -1.000000Do time calibration: TrueMax iterations: 30Time offset padding: 0.030000Estimating time shift camera to imu:Initializing a pose spline with 0 knots (100.000000 knots per second over 0.000000 seconds)Traceback (most recent call last):File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>exec(compile(fh.read(), python_script, 'exec'), context)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>main()File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 189, in mainiCal.buildProblem(splineOrder=6, File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 105, in buildProblemcam.findTimeshiftCameraImuPrior(self.ImuList[0], verbose)File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 221, in findTimeshiftCameraImuPriorposeSpline = self.initPoseSplineFromCamera( timeOffsetPadding=0.0 )File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 315, in initPoseSplineFromCamerapose.initPoseSplineSparse(times, curve, knots, 1e-4)RuntimeError: [Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/bsplines/src/BSpline.cpp:970: initSplineSparse() assert(numSegments >= 1) failed [0 >= 1]: There must be at least one time segmentroot@e9491cc5700e:/catkin_ws#

rosrun kalibr kalibr_camera_validator --target ../data/april6.yaml --cam ../data/Stereo-camchain.yaml rosrun kalibr kalibr_calibrate_cameras --models pinhole-equi pinhole-equi --topics /camera/left/image_raw /camera/right/image_raw --bag src/kalibr/datakalibr_calibrate_cameras --models pinhole-equi pinhole-equi --topics /cam0/image_raw /cam1/image_raw --bag static.bag --target aprilgrid_6x6.yamlrosrun kalibr kalibr_calibrate_cameras \--bag /data/cam_april.bag --target /data/april_6x6.yaml \--models pinhole-radtan pinhole-radtan \--topics /cam0/image_raw /cam1/image_raw

Skip to content

Search or jump to…

Pull requests

Issues

Marketplace

Explore

@KOBOSP

ethz-asl

/

kalibr

Public

Code

Issues

80

Pull requests

8

Actions

Projects

Wiki

Security

Insights

Calibrating the VI Sensor

Timo Hinzmann edited this page on 26 May · 19 revisions

Pages 17

Home

Downloads

Installation

Toolbox Tools

Multiple camera calibration

Camera-IMU calibration

Multi-IMU and IMU intrinsic calibration

Rolling Shutter camera calibration

(only ROS):

Camera focus

Calibration validator

ROS2 support

Formats

Supported camera models

Calibration targets

Bag format

YAML formats

IMU Noise Model

Tutorials

Example: Calibrating a VI-Sensor

Experimental

Camera-IMU-LRF calibration

Clone this wiki locally

/ethz-asl/kalibr.wiki.git

VI-Sensor

This page will guide you through the calibration of the VI-Sensor (visual-inertial sensor). The intrinsics and extrinsics of the camera system and the transformation of each camera w.r.t. the IMU will be estimated.

More information about the VI-Sensor can be found here.

Procedure

prepare the sensor

setting the focus

collect calibration data

in-/extrinsics calibration (static calibration)

imu-camera calibration (dynamic calibration)

run the calibration

in-/extrinsic camera calibration

imu-camera calibration

collect results

Requirements

ROS sensor driver is running (image/imu data)

good Aprilgrid target (pdf, yaml)

Siemens star (or similar camera focus test pattern)

IMU configuration for ADIS16448 (yaml)

Sensor preparation

make sure the sensor publishes all image and imu streams to ROS

minimize the motion blur with a good light source and by reducing the shutter times.

Shutter times can be set using the following commands:

rosrun dynamic_reconfigure dynparam set /visensor_node “{‘cam0_agc_enable’: 0, ‘cam0_aec_enable’: 0, ‘cam0_coarse_shutter_width’: 300}”

rosrun dynamic_reconfigure dynparam set /visensor_node “{‘cam1_agc_enable’: 0, ‘cam1_aec_enable’: 0, ‘cam1_coarse_shutter_width’: 300}”

Observe the result on an image window and tweak the shutter until you get a good image:

rosrun image_view image_view image:=/cam0/image_raw &

rosrun image_view image_view image:=/cam1/image_raw &

Setting the focus

point the cameras on a Siemens star (or similar pattern)

start the focus tool

kalibr_camera_focus --topic /cam0/image_raw /cam1/image_raw

set the focus of both cameras by:

reducing the interference visible around the center of the Siemens star

minimizing the focus measure provided by the tool

Make sure a Teflon band or thread-locking glue prevents unintentional focus changes after this step.

Collect calibration data

In this step we need to collect two calibration datasets with the following properties:

static dataset (in-/extrinsic calibration of the cameras)

attach the sensor somewhere and move the target

limit the camera streams to ~4 Hz

make sure to cover the entire field of view of the camera

use skewed views and varying distances to the calibration target

view images with:

rosrun image_view image_view image:=/cam0/image_raw &

rosrun image_view image_view image:=/cam1/image_raw &

record bag with:

rosbag record /cam0/image_raw /cam1/image_raw -O static.bag

dynamic dataset (spatial camera-imu calibration)

move the sensor (target is fixed)

cameras should run at 20 Hz and IMU at 200 Hz

try to excite all rotation and acceleration axes of the IMU

avoid shocks (e.g. while picking up the sensor)

good illumination and shutter times are crucial here (to avoid motion blur while exciting the IMU)

view images with:

rosrun image_view image_view image:=/cam0/image_raw &

rosrun image_view image_view image:=/cam1/image_raw &

record bag with:

rosbag record /cam0/image_raw /cam1/image_raw /imu0 -O dynamic.bag

Run the calibration

calibration of camera in/extrinsics

run calibration

kalibr_calibrate_cameras --models pinhole-equi pinhole-equi --topics /cam0/image_raw /cam1/image_raw --bag static.bag --target aprilgrid_6x6.yaml

inspect the result plots

verify calibration on the live image stream

reprojection errors should be in a normal range (0.1-0.2 px for a good calibration)

kalibr_camera_validator --chain chain.yaml --target aprilgrid_6x6.yaml

camera-imu calibration

run calibration

kalibr_calibrate_imu_camera --cam chain.yaml --target aprilgrid_6x6.yaml --imu imu0.yaml --bag dynamic.bag

inspect the result plots

make sure the predicted accelerations & angular velocities fit the IMU measurements

reprojection errors should be in a normal range (0.1-0.2 px for a good calibration)

4) Collect results

Both calibrators write reports to the working directory containing the plots shown at the end of the calibration. Further a camchain.yaml has been written by the camera calibrator and is extended by the imu-camera calibrator with imu-camera transformations to the file camchain_cimu.yaml. Please refer to the YAML formats page for the format.

Footer

© GitHub, Inc.

Footer navigation

Terms

Privacy

Security

Status

Docs

Contact GitHub

Pricing

API

Training

Blog

About

You have no unread notifications