Skip to content

Related Work

Research

The HTC VIVE Tracker can be used to enable position tracking and transmission of specific data. It can be a good “development kit” for developing VR controller. Therefore, recent works, which utilize the Tracker, for making a VR controller were reviewed for establishing design requirements.

https://developer.vive.com/eu/vive-tracker-for-developer

The idea of the VIVE Tracker is to go beyond controller. It can be incorporated into specially designed accessories for various XR applications (e.g. guns, gloves, or DSLR cameras).

https://www.vive.com/eu/vive-tracker/

Note

Firmware has been released to add USB input and it’s been officially released in the Stemar beta branch. There’s even an updated Tracker puck 2018 1.4.

As a start point, we reviewed existing Works in Literature:

  • Google Scholar Database.
  • Review works since 2018 only (The HTC VIVE Tracker v.2 was released in 2018).
  • Research was conducted on 15th May 2020.
  • Search term: HTC “Vive” Tracker “controller” design.
  • Target designs and implementations that utilize the HTC VIVE Tracker for 1) controller tracking in VR and/or 2) communicating with PC.
  • we focus on handheld VR controllers that augment the typical controller functionality, but we also include glove and mountable designs.
  • We focus on works with a specialized designs and implementations that add to/increase the typical controller functionality.
  • We exclude works where the controller is simply the Tracker; these are usually simple designs that are limited to the tracker’s functionality and do not participate to the targeted experience in our controller’s case. Additionally, such works do not utilize the tracker in the desired way that satisfies the Fab Academy final project requirements. E.g., simply mounting the tracker to a stick or tennis racket)
  • We avoid including works that use the HTC VIVE Tracker v.1 since it is discontinued.

The selected works can be categorized in a chronological order. We summarize the design and implementation for each work. We also identify which HTC Tracker use case was used:

2020

“Xavier de Tinguy, Thomas Howard, Claudio Pacchierotti, Maud Marchal, Anatole Lécuyer. WeATaViX: WEarable Actuated TAngibles for VIrtual reality eXperiences. EUROHAPTICS 2020 - 12th International Conference on Haptics, Sep 2020, Leiden, Netherlands. pp.1-8.”

THE prototype is composed of a 3D-printed structure to be placed on the back of the hand. Its profile is slightly curved to fit the shape of the hand. On the internal side, it is anchored in an adhesive silicone skin.

An HTC Vive Tracker can be attached on the external side. The distal side of the 3D-printed structure houses a HiTec HS-5065MG servomotor which controls the motion of a rigid link holding the tangible object. By moving the rigid link, the motor brings the tangible object towards or away from the user’s palm. The tangible object is equipped with capacitive sensors to detect contacts with the hand.

The HTC Vive tracker enables hand position tracking and, together with the capacitive sensors, animation of the user’s hand avatar in VR. Haptic device composed of a 3D-printed part anchored to an adhesive silicone layer attached to the hand. Two capacitive sensors cover the tangible, respectively facing the palm and the fingers during grasp closure.

(A) Schematic of the interconnected electronics structure for sensing and con-trol. The capacitive sensing uses the Arduino CapacitiveSensor library. (B) VR setup.

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to interact with the Tracker. This is a simple design. However, it was interesting due to the design of ‘Tangible Object’ and the inclusion of capitative sensors. #Straps

“Zenner, André, et al. “Demonstration of Drag: on-A VR Controller Providing Haptic Feedback Based on Drag and Weight Shift.” Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts. 2020.”

A shape-changing VR controller Drag:on, which leverages the airflow that occurs at the controller during interaction to provide a range of different haptic sensations. The device can increase or decrease its surface area and by this also adapts its mass distribution. For this, Drag:on uses actuators only to change the physical configuration of the device to change how it feels when moved through the air.

Different states of the Drag:on controller that participants can experience in this demonstration, differing in surface area and weight distribution.

The device is connected to a controller box containing an Arduino Nano microcontroller and the necessary circuits. An external power adapter connects to this box to provide 7.6Vto the motors. The Arduino can interface with a PC via USB serial communication. Drag:on can increase its surface area by up to 650%.

The 3D rendering shows the main components of the Drag:on device. The actuation mechanism depicted in the bottom image of Figure is located at the top end of the controller. Both sides hold an MG996R servo motor using custom 3D-printed parts, each actuating a 3D-printed arm attached to the topmost layer of a commercially avail-able flamenco hand fan. The fans are 31cm long and made out of wood and fabric. The bottommost layer of the fan is rigidly attached to a 3D-printed support structure. By actuating the servo, the arm opens or closes the fan.

Note

Again, the Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to interact with the Tracker. However, it is interesting due to the design of ‘Fans’. #Handheld #Shape/weight shifting.

“Rahimi, Ali, Jackson Zhou, and Sasan Haghani. “A VR gun controller with Recoil Adjustability.” 2020 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2020.”

They design and implement a VR gun controller with haptic feedback for the HTC Vive with recoil adjustability. The primary focus of this design is to provide realistic haptic feedback for games utilizing projectile launching weapons to improve gaming immersion on a level further than simply audio and visual.

The player creates certain actions in a scene through a button press that will elicit a haptic response. The Vive Tracker sends a command to the VR scene and the user observes the response through the HMD.

Note

This one is one of its kind! The USB interface was used to interact with the Tracker v.2. The STM32L476 acts as a USB Host. Payload packets are sent to the tracker. #Handheld #Haptic

2019

“Liu, Hangxin, et al. “High-fidelity grasping in virtual reality using a glove-based system.” 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019.”

They present a design to enable a caging-based natural grasp in the VR. The hardware design is based on an easy-to-replicate glove that senses hand pose through a network of 15 IMUs, localizes hand using a Vive Tracker, and provides haptic feedback through 6 vibration motors. We have demonstrated that the design captures fine-grained hand motions. According to the collision’s geometry between the virtual hand and virtual objects, a caging-based approach is integrated to determine a stable grasp.

Warning

This one uses the Tracker v.1.

A Vive Tracker and a network of IMUs provide comprehensive hand pose and location in the VR simulator Unreal Engine 4 (UE4). When a stable grasp is formed based on the contact geometry, the virtual object follows the hand’s movement, and the vibration motors at the contact area are triggered to provide the user with vibrational feedback.

The prototype consists of a network of 15 IMUs, a network of 6 vibration motors, and a Vive Tracker.

Note

The Vive Tracker was used for hand localization, and a network of vibration motors to provide vibrational haptic feedback. A light-weight Raspberry Pi serves as the control unit of the system that collects and transmits the data between the physical glove and the virtual environment. #Glove #Haptic

“Burdea, Grigore, et al. “Assistive game controller for artificial intelligence-enhanced telerehabilitation post-stroke.” Assistive Technology (2019): 1-12.”

They present a design to enable a caging-based natural grasp in the VR. The hardware design is based on an easy-to-replicate glove that senses hand pose through a network of 15 IMUs, localizes hand using a Vive Tracker, and provides haptic feedback through 6 vibration motors. We have demonstrated that the design captures fine-grained hand motions. According to the collision’s geometry between the virtual hand and virtual objects, a caging-based approach is integrated to determine a stable grasp.

The tracker was chosen due to its small dimensions, ability to provide 120 data sets every second, wireless transmission, and easy integrations with the HTC VIVE simulation system.

It provided tracking data for the simulation, allowing the player to move objects in the virtual world. The actual tracking was based on infrared (IR) beams transmitted by two VIVE IR emitters (part of the BB system) and detected by an array of IR sensors integrated in the VIVE tracker.

The tracker was mounted with regular DSLR camera mount (¼–20 tripod) screws on top of a custom support.

Note

Again, the Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to interact with the Tracker. However, it is interesting due to the use of a magnetic rotary sensor. #Straps/Mounted #Shape/weight shifting.

“Glowacki, Becca Rose, et al. “An open source Etextile VR glove for real-time manipulation of molecular simulations.” arXiv preprint arXiv:1901.03532 (2019).”

They describe Open-source Mudra Gloves for VR (OMG-VR): etextile gloves designed to facilitate research scientists and students carrying out detailed and complex manipulation of simulated 3d molecular objects in VR.

The HTC vive tracker attaches to 3d printed mount that houses pogo pins. Pins connect tithe silver plated fabric.

Note

The Tracker was used for Tracking and sending digital input data! The Pogo pins were used. #Glove

“Shigeyama, Jotaro, et al. “Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019.”

They propose Transcalibur, which is a hand-held VR controller that can render a 2D shape by changing its mass properties on a 2D planar area. We built a computational perception model using a data-driven approach from the collected data pairs of mass properties and perceived shapes. This enables Transcalibur to easily and effectively provide convincing shape perception based on complex illusory effects.

Transcalibur is controlled by Teensy 3.2 with ARM 72MHzCortex-M4 MCU, programmed by Arduino IDE. Each motor in each mechanism is driven by DRV8871 motor drivers and is connected to the MCU. The device is powered by DC voltage of 12 V, and a regulated voltage of 5 V is supplied both to the MCU and the motor drivers. MCU and motor drivers are mounted on printed circuit board (PCB) and stowed in3D printed PLA handle of Transcalibur. Position data can be transferred by both USB-serial communication and electric signals that can be obtained from Pogo pins connected to the HTC VIVE Tracker. The position or angle of mechanisms is monitored and controlled by magnetic encoders attached to the motors of each mechanism.

Transcalibur can dynamically present size of various object in VR, actuating weight and angle mechanisms. Although the actual controller’s appearance differs from its appearance in VR, a user feels as if s/he is wielding a sword, a gun and a crossbow with the same VR controller

Note

The Tracker was used for Tracking and sending digital input data! The Pogo pins were used. #Handheld #Shape/weight shifting.

“Yi, HyeonBeom, et al. “DexController: Designing a VR Controller with Grasp-Recognition for Enriching Natural Game Experience.” 25th ACM Symposium on Virtual Reality Software and Technology. 2019.”

They present DexController, which is a hand-held controller leveraging grasp as an additional modality for a VR game. The pressure-sensitive surface of DexController was designed to recognize two different grasp-poses (i.e. precision grip and power grip) and detect grasp-force.

The sensor layer was surrounded on the body of acryl cylinder. The Arduino detected the change of resistor between the copper tapes and calculated the pressure data matrix. For a passage of wires, there was a 4mm vertical blind region that cannot detect pressure.

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to interact with the Tracker. However, it is interesting due to the use of a pressure sensor. #Handheld.

“Sinclair, Mike, et al. “CapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback.” Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 2019.”

CapstanCrunch is a haptic controller that renders touch and grasp haptic sensations in virtual reality. CapstanCrunchsupports human-scale forces during touch and grasp and can resist the user’s grasp force up to 20N.The friction-based capstan brake mechanism magnifies the small motor’s force, resulting in an integrated, palm-grounded controller design for interaction. The controller can render complex haptic events exhibiting variable stiffness and compliance.

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to send/receive packets to the Tracker. However, it is interesting from a robotics/ mechanical perspective. It is based on CLAW, reviewed next. #Handheld.

2018

“Choi, Inrak, et al. “Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality.” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018.”

They prototyped a handheld VR controller. In addition to the typical controller functionality, force feedback and actuated movement to the index finger are included. Haptic rendering is based on sensing the differences in the user’s grasp. The controller allows for three distinct interactions:

  • Grasping a Virtual Object
  • Touching a Virtual Surface
  • Triggering

Controllable forces to the index finger during grasping and touching are rendered using a servo motor and a force sensor. Vibrations, for various textures, synchronized with finger movement are generated using a voice coil actuator at the index fingertip. Additionally, CLAW allows for haptic force feedback in the trigger mode when the user holds a gun. CLAW’s performance is evaluated through two user studies:

  • Qualitative user feedback on the naturalness, effectiveness, and comfort when using the device
  • Investigated the ease of the transition between grasping and touching

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to send/receive packets to the Tracker. However, I personally like this one from a robotics/ mechanical perspective. A lot of innovation here! It is based on CLAW, reviewed next. #Handheld. #Haptic

CLAW VR controller provides articulated movement and force feedback actuation to the user’s index finger which allows for convincing haptic rendering of: (a) grasping, (b) touching, © rendering virtual textures, and (d) triggering.

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to send/receive packets to the Tracker. However, I personally like this one from a robotics/ mechanical perspective. A lot of innovation here! It is based on CLAW, reviewed next. #Handheld. #Haptic

“Mac Murray, Benjamin C., et al. “A variable shape and variable stiffness controller for haptic virtual interactions.” 2018 IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 2018.”

They presented an entirely compliant controller handle for use in VR. The controller handle transitions between two static states:

  • A semi-rigid, large diameter state when pressurized
  • A soft, compressible, smaller diameter state when depressurized

They integrated the controller with a modified version of NVIDIA’s VR Funhouse (a VR App). The two controller states are used to simulate the physical feel of two virtual objects.

Controller prototype system integration schematic. They leveraged the HTC Vive tracker’s integration with the SteamVR runtime. A firmware switch caused the tracker to appear as a normal controller to the application with full lighthouse-based tracking. NVIDIA’s VR Funhouse was modified to send commands to a python script connected over a USB cable to an Arduino microcontroller that operated the valves. Furthermore, we added a virtual model of a foam sword using a soft-body physics simulation to the VR Funhouse environment.

Note

The Tracker was used for Tracking. The Pogo pins were used. #Handheld #Shape/weight shifting.

“Whitmire, Eric, et al. “Haptic revolver: Touch, shear, texture, and shape rendering on a reconfigurable virtual reality controller.” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018.”

A handheld VR controller that renders fingertip haptics when interacting with virtual surfaces is presented by Whitmire and colleagues. The haptic sensation is realized using an actuated wheel, which raises and lowers underneath the finger, to render contact with virtual surfaces. The wheel is interchangeable to provide different sensations to the user.

The device is powered by a Programmable System-on-Chip (PSoC) that communicates with two motor drivers and a PC via a USB serial connection-

They evaluated the controller in two studies to see how the wheel speed and direction could impact the perceived realism in three application scenarios, where users provide qualitative feedback.

Note

The Tracker was used for Tracking in VR only. Neither of the Pogo pins nor the USB interface were used to send/receive packets to the Tracker. Another robotics/ mechanical perspective. #Handheld. #Haptic

Summary

  • Use Tracker for tracking VR.

  • Use the Tracker’s USB interface to send/receive packets to/from the Tracker v.2. (Would be great for Analog!). If this fails :(, go with Pogo pins (digital only).

* #Glove * #Straps/Mounted * #Shape/weight shifting

* #Handheld * #Haptic