| abstract |
A system includes a near eye display (NED) that comprises an optical assembly with an electronic display, an imaging sensor configured to capture images of a user's hands, and an eye imaging sensor configured to capture images of an eye of the user. The system also includes a controller configured to determine eye tracking information using the captured images of the eye, the eye tracking information indicating a gaze orientation, wherein the gaze orientation terminates at first location. The controller determines that a pose of the user's hand indicates a pinch gesture based on the captured images of the user's hands. The controller also identifies an object in the local area that is at the first location, and updates the display instructions to cause the electronic display to display an indication of a selection of the object in an artificial reality environment. |