Non-Contacting Two-Dimensional Position Estimation Using an External Magnet and Monocular Computer Vision¹

Accurate 2D position estimation systems are valuable in today's world of increased automation and research studies. An example application of an accurate 2D position estimation system is rodent tracking in animal studies. There are several methods available for animal tracking, including both “contacting” and “non-contacting” methods. “Contacting” methods for trajectory tracking involve wired sensors attached to the animals [1,2]. On the other hand, “non-contacting” methods utilize cameras [3,4]. Unlike them, this work aims to use depth information in addition to monocular camera vision to achieve submillimeter-level accuracy in real-time object/animal position estimation.

Position estimation solely through the use of a monocular camera is a difficult task. The theoretical feasibility of using a monocular camera for position estimation has been discussed in Refs. [5,6]. Using a monocular camera for position estimation compared to stereo vision involves less data and also eliminates the need for synchronization and stereo calibration. However, accurate depth estimation from a single 2D image is an extremely challenging problem. With rapid improvements in the field of machine learning, even though a variety of deep learning networks have been successful in-depth estimation from monocular camera images [7,8], their accuracy is limited to the order of meters. Further, training deep neural networks for such distance estimation is a tedious task. This paper concentrates on tracking animals/robots with millimeter-level accuracy on a two-dimensional plane.

This work utilizes an additional sensor along with a monocular camera for accurate depth estimation. Here, a magnetic sensor is mounted on the animal/robot to be tracked and an external permanent magnet generates a magnetic field. Position estimation using a magnetic field has previously been carried out by utilizing the magnetic field generated by the permanent magnet [9–11] or coils [12,13]. The advantage of permanent magnets is that the magnetic source is power-free, and the magnetic field strength is significantly higher. But their use is limited to environments where magnetic field disturbance is minimal. This paper presents a method for actively controlling the orientation of the permanent magnet using visual feedback from a monocular camera, in order to accurately point it at the animal or robot. By doing so, it enables the estimation of the animal or robot's 2D position with millimeter-level accuracy, using a simple 1D magnetic field model instead of a more complex 2D magnetic field model of the permanent magnet. The radial range of the position estimation is up to 60 cm, which is relevant for many practical applications. For the purposes of this paper, we will focus on tracking a robot; however, it should be noted that this method can also be used to track any other object or animal on a 2D plane

A schematic diagram for the estimation system is shown in Fig. 1. The active measurement system consists of a permanent magnet mounted on a stepper motor, a low-cost camera mounted on top of the rotating permanent magnet, a small spherical marker on the robot to enable object detection with the help of the camera, and a two-axes magnetic field sensor chip installed on the moving robot. The importance of adding a low-cost camera to the estimation system compared to the estimation system developed in the previous work [14] will be discussed in the section “Position Estimation Principles”. This work assumes that the robot motion is limited to a 2D XY plane. The robot will have three DoF as shown in Fig. 1(b).

The permanent magnet creates a strong magnetic field at the robot location. The magnetic field at the robot location is sensed along two mutually perpendicular axes on the robot. The analog signal output from the magnetic sensor is amplified using an instrumentation amplifier chip (INA2126) and acquired using a data acquisition device. A low-cost USB camera (ELP megapixel Super Mini 720p USB Camera) with a frame rate of 30 fps is used for live capturing of robot motion.

A robust 1D radial position estimation system is developed using the magnetic field strength measured at the robot position and a nonlinear observer-based position estimation algorithm. In polar coordinates, the magnetic field at the robot location (ρ, θ, z) is a function of ρ, θ, and z (Fig. 3). The vertical offset z is a known constant parameter (≈0 cm) in this 2D estimation problem. The permanent magnet is mounted on a NEMA 23 stepper motor so that its orientation can be controlled in real time. The permanent magnet is continuously pointed at the magnetic sensor using an active control algorithm with the help of visual feedback from the USB camera. Hence, θ of the position coordinate of the robot with respect to the magnet will be zero due to active control and helps to remove the θ dependence of the magnetic field at the robot location. Thus, the 2D position estimation problem is reduced to a 1D position estimation problem by using active orientation control i.e., the magnetic field at the robot location will be a function of just the radial distance ρ. The polar angle θ is estimated from the encoder readings of the stepper motor, and the radial distance of the robot from the magnet ρ is estimated from the magnetic field measurements.

The components of the magnetic field due to the magnet, $Bx′andBy′$⁠, sensed by the magnetic sensor at a position vector $r⇀(r⇀=ρ⇀+z⇀)$ are determined by the parameters α, θ, and γ shown in Fig. 3. Here, the variables in the figure can be described as follows:

• x, y: global frame of reference for robot position.

• x′, y′: magnetic sensor frame of reference.

• α: angle subtended by the magnet to the x-axis.

• γ: angle subtended by the magnetic sensor to the x-axis.

• $ρ⇀$⁠: the position vector of the magnetic sensor with respect to the magnet in the XY plane.

• θ: angle subtended by the position vector, $ρ⇀$ to the x-axis.

• $m⇀$⁠: magnetic dipole moment vector of the magnet.

The 2D position estimation system using only a single magnetic field sensor developed in the previous paper [14] solved the estimation problem using the following lemmas:

The previous 2D position estimation system discussed earlier utilizes Lemma 1 and develops an active orientation control algorithm for pointing the magnet at the robot continuously. As per Lemma 1, the magnetic field is maximum when the magnet is pointed at the robot, i.e., when α = θ. The control algorithm involves a search of the α at which the $|BH⇀|$ (magnitude of the magnetic field at the robot location) is maximum. The feedback error term used for the control algorithm is the rate of change in the magnitude of the magnetic field with respect to the change in the magnet angle $(d|BH⇀|dα)$⁠. The variation of the magnetic field strength at the robot location $(|BH⇀|)$ with (α − θ) is shown in Fig. 4. Since, $d|BH⇀|dα→0$ as α converges to θ, the pointing angle of the magnet needs to keep on oscillating continuously (+/−1 deg) to find the direction in which the magnetic field changes when the robot is in motion. And also, the bandwidth of the pointing control was limited due to the need for back-and-forth searching. For better accuracy in position estimation, it is necessary to have a better-pointing angle control system. One of the objectives of the current work is to incorporate an inexpensive camera that can provide visual feedback for eliminating the need for magnet oscillation while accurately pointing the magnet at the robot.

The current work is based on sensor fusion of computer vision and magnetic field-based measurements for position estimation. This sensor fusion provides some obvious advantages compared to that of Ref. [14], which only used magnetic sensor measurements. The introduction of computer vision helps to remove the need for constant searching of the direction of robot/sensor motion; hence, complex high bandwidth motions can be tracked much more effectively and continuously (which is essential for animal tracking applications). Computer vision helps with better pointing control; hence, pointing control can be applied at farther distance ranges as clearly demonstrated in the future sections of this paper. The previous work [14] had a radial distance range of only up to 20 cm whereas the radial distance range of the current work is tripled, i.e., increased to 60 cm which helps to serve many more applications compared to the previous work. Current work achieves this with the addition of a low-cost USB camera to the two-axes magnetic sensor and using a permanent magnet instead of an electromagnet.

In order to utilize visual feedback for orientation control, robust object detection and tracking are required for tracking the spherical marker. An OpenCV-based CSRT (Channel and Spatial Reliability Tracking) tracker is used for object detection and tracking. CSRT tracker is a C++ implementation of the CSR-DCF (Channel and Spatial Reliability of Discriminative Correlation Filter) [15]. In the first frame, the user needs to mark the bounding box around the marker, and then, the CSRT tracker will track the marker as time continues. The CSRT tracker only enables the bounding box detection around the marker. In many cases, the center of the bounding box may not align with the center of the spherical marker. Hence an OpenCV-based circle detection using the Hough transform.³ is used to detect the spherical marker inside the bounding box in order to estimate the center of the marker precisely. Target tracking results from multiple scenes using OpenCV are provided in Fig. 7. The scenes vary from as close as 5 cm to as far as 60 cm. The figure shows the CSRT tracker is able to locate the spherical marker in a wide range of motion, and the circle detection algorithm is able to detect the marker in the bounding box. Thus, accurate live tracking of the spherical marker on the robot/sensor improves the accuracy of the orientation control and eliminates magnet oscillations while pointing at the robot/sensor.

Another limitation of the 2D position estimation method presented in the paper [14] is its limited radial distance range. The range of radial distance estimation was only 20 cm, which is insufficient for many practical applications. The magnetic field source of the above system [14] was an electromagnet. One possible solution to increase the range is to increase the current passing through the electromagnetic coil, which would result in a stronger magnetic field. However, increasing the current also leads to increased heat dissipation from the electromagnetic coil, which will increase the resistance of the coil. As a result, the magnetic field at a certain position coordinate (x, y) may not be solely a function of the position coordinate but a function of the temperature of the coil as well. The current research in this paper is exploring the use of a permanent magnet to generate the magnetic field instead of an electromagnet. Permanent magnets generate a constant, time-invariant field, and a strong permanent magnet helps to extend the radial distance range of position estimation. Thus, using a permanent magnet instead of an electromagnet helps in enhancing the range of the radial distance estimation.

The 2D position estimation problem consists of two main components:

• Active rotation control of the magnet to point at the robot;

• Estimation of the radial distance of the robot from the magnet using a nonlinear observer.

The unknown jerk can be considered to be zero in a deterministic estimation framework or as zero mean Gaussian noise in a stochastic framework. It should be noted that the use of a zero radial jerk model underlies an assumption that the acceleration changes slowly (i.e., its derivative is small).

Note that the estimation error dynamics Eq. (8) are nonlinear, requiring a nonlinear observer design [16]. Theorem 1 below describes how to choose L to stabilize this nonlinear system.

This section discusses the experimental results of the developed position estimation system. The developed 2D position estimation system consists of active orientation control and radial distance estimation. Three experiments are discussed to demonstrate the efficiency of the estimation system. In the first experiment, the robot is kept at an arbitrary position as shown in Fig. 8, and the active control algorithm manipulates the permanent magnet for pointing at the robot. The plot shown in Fig. 9 confirms that the control algorithm is able to control the orientation of the magnet and continuously point at the robot without any oscillations. Experimental results of the radial distance estimation of the robot from the magnet are provided in Fig. 11. In this experiment, the robot is initialized at a random position and then moved along the radial direction as shown in Fig. 10 (the robot is moved from point A to point C along the radial direction). The active control algorithm enables the magnet to continuously point at the robot and estimate the radial distance of the robot from the magnet. The error in the radius estimation is provided in Fig. 12. The error is computed using a reference distance measurement obtained from a laser displacement sensor (Banner Engineering laser displacement sensor). The magnet angle (α) is estimated from the encoder readings of the motor on which the magnet is mounted. The magnet angle estimation results during the above experiment are presented in Fig. 13. It can be seen that radius estimation is achievable with submillimeter accuracy (Fig. 11) by using the magnetic field and a nonlinear observer. As previously discussed, the active control algorithm enables the continuous pointing of the magnet at the robot. In the above experiment, the robot is traveling along a straight line at an angle (θ) of 0 deg from the origin. The results in Fig. 13 indicate that the active orientation control algorithm is able to control the magnet angle (α) to continuously point at the robot (α = θ) with an accuracy of +/−0.5°. And also, the range of the radial distance estimation is nearly 60 cm.

In the previous experiment, the robot traveled along a straight line. In the next experiment, the robot travels along a curved path. The simultaneous position estimation results during this robot movement are provided in Fig. 14. Figure 15 shows the error in the radial distance estimation, and Fig. 16 shows the magnet angle estimation results. These results demonstrate that the developed position estimation system is capable of continuously pointing the magnet at the robot and accurately estimate the position of the robot while it is undergoing a curvilinear motion.

This paper presents the development of a 2D position estimation system using a permanent magnet, magnetic field feedback, and visual feedback from a monocular camera for an object traveling in a 2D plane. The developed estimation system has several advantages including low-cost, non-contacting operation, easy installation, and the use of a simple 1D magnetic field map for position estimation instead of a complex 2D magnetic field map. In this estimation technique, the magnet is constantly pointed at the robot using a proportional controller based on visual feedback from the USB camera. Then the radial position of the robot is estimated using a nonlinear observer and the magnet angle is estimated from the encoder data of the motor on which the magnet is installed. The results demonstrate that the radial distance of the robot from the magnet can be estimated with submillimeter accuracy and the angle at which the robot is positioned with respect to the origin can be estimated with an accuracy of 0.5 deg. Furthermore, the maximum range of the position estimation system extends up to 60 cm from the magnet.

This work was funded in part by a research grant from the National Science Foundation (NSF Grant EFMA 1830958).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.