Image modeling algorithm for environment design based on augmented and virtual reality technologies

2.1 Image modeling

Because image reconstruction needs to use different methods according to different image types, numbers, and reconstruction methods, there is no unified processing flow for image-based 3D reconstruction.

Figure 1 shows the basic flow of the image-based 3D reconstruction method.

Figure 1

Basic flow of the image-based 3D reconstruction method.

It can be seen from Figure 1 that, first, multiple sample images are used as input for calibration to obtain the direction and camera sampling parameters related to each sample image and 3D scene. Second, the point cloud data of the target surface is reconstructed in 3D. Third, the surface of the target is restored to obtain the reconstructed 3D geometric model. Finally, the texture of the object surface is processed to obtain the required 3D model.

In the method of image modeling based on the environment design, the geometric model of the scene or object should be reconstructed first, and then the visual effect of the scene should be obtained from the following two aspects. The image modeling method based on environment design carries out texture extraction and texture mapping on the reconstructed geometric model to generate a surface with high fidelity, thus avoiding the tedious lighting operation in the traditional drawing process and solving the problem that has been puzzling users’ rendering speed and realism for a long time. The image modeling method based on the environment design can obtain the reflection characteristics and light conditions of the surface, which provides the most intuitive basis for the rendering of the model, so as to achieve the real effect that cannot be achieved under the manually set reflection properties and light conditions. In addition, it can be rendered realistically in the case of light changes.

2.2 AR and VR technologies

In recent years, VR technology has been widely used in various fields but it also exposes its inherent problems, such as the isolation of the objective environment, which makes people feel unreal [11,12]. AR technology is an effective way to solve the above problems. It integrates virtual objects or other information generated by computers (such as text and illustration) with the objective environment to strengthen the objective world [13].

AR technology is the product of VR technology, but there are essential differences between them [14]. VR technology emphasizes that users are not in conflict with the real environment but are attracted by the virtual world created by computers. The characteristic of AR technology is that it is not isolated from the real environment but also can highlight the existence of the objective world and can improve the user’s perception ability [15]. AR technology pursues the unity of “virtual” and “real,” and “virtual” and “real” complement each other. At the same time, because the relationship between users and the real world has not been completely separated, their interaction is more harmonious and natural. AR technology builds a bridge between VR and reality and can realize many interactions that cannot be realized in reality. Its application prospect is very broad. Today, with the development of human society, this technology has been widely used in all walks of life.

VR technology is a virtual world created by people and computers. It can obtain information from multiple users in real time and track and interact with them in real time [16,17]. “Virtual” is generated by a computer using 3D software and programming language, and it does not exist. “Reality” is the physical space that can be felt and seen. VR technology is a comprehensive and cutting-edge technology that covers computer, human–computer interaction, image art, body feeling technology, and many other professional technologies [18,19]. With humans as the core, VR technology uses human perception technology to capture the user’s behavior, position and vision, compile them into data, and transmit them to the database [20]. The database re-enters the real-time user information into the virtual image and modifies it through real-time user data. This leads to changes according to the user’s actions when the user makes an action. Compared with the previous “Windows,” the computer image can be operated by a mouse, keyboard, and other tools [21].

The environment design image modeling algorithm model based on AR and VR technologies is composed of the scene interaction module, 3D model rendering module, and AR module. Figure 2 shows the main functions of the system [22].

Figure 2

Main functions of the system.

It can be seen from Figure 2 that the main functions of the system include simulation of real scenes, dynamic interaction, and real-time rendering. Among them, the 3D model building module models the shape of the virtual object [23]. Taking the building in the real scene as an example, the trees and plants outside the building, and the walls and ceilings inside the building are modeled [24]. Based on the 3D model, the virtual and real environment effect is established. By calling multiple layers in geographic information systems (such as topographic maps and building outline maps) and Line View technology, the fusion and interaction between geometry and virtual objects in real scenes are realized. This module can realize the digital representation of scenes in real scenes [25].

2.3 Implementation of image modeling algorithm for environmental design

In this study, the complex virtual scene is divided into two categories: regular shape and irregular shape. To solve the problem of modeling complex irregular objects, a virtual scene modeling method based on multi-view photo sequence is proposed. The basic flow of the algorithm is shown in Figure 3.

Figure 3

Basic flow of the algorithm.

It can be seen from Figure 3 that, first, according to the specific accuracy, the complex objects modeled are extracted from photos from multiple angles. On this basis, the positions of photos and observation points are combined to form a one-to-one “photo-view” relationship. Geometric virtual scene modeling technology is applied to rapidly generate regular shape models and other scene components included in the virtual environment. Finally, the composite virtual model generated by the photo sequence is used to integrate it into the virtual scene.

When the viewer roams in the virtual environment, it can track the position of the observation point in the world coordinate system in real time and convert it into the target coordinate system. By defining the corresponding relationship between “photo and perspective,” the picture corresponding to the current perspective is determined and displayed to the viewer until the user leaves the roaming scene.

During shooting, the photographic background corresponding to the surface material, color, and other characteristics of the object should be selected as much as possible, and the appropriate brightness and number should be set. The image sequence of the modeled object from different angles is obtained and then segmented. The background of the photo is separated from the outline of the object to form a series of transparent backgrounds, which is used for the next step of modeling. During the acquisition process, a background with obvious characteristics can be selected according to the surface material, color, and other characteristics of the modeled object, thus simplifying the image content and improving the image segmentation effect.

With the continuous development of image-matching technology, image-matching technology came into being. The current feature-based methods are basically based on the characteristics of the image itself, and the region-based matching algorithm is based on this [26].

Association degree operation is a commonly used matching algorithm at present, which is mainly realized by various correlation degree functions. Among them, the simplest correlation operation is completed by the average value of the brightness difference between pixels. Some researchers also proposed a correlation operation based on standard deviation correlation operation and divided it into low-level processing and high-level processing.

First, the gradient-based Plessey corner extraction algorithm is adopted. In the original gray image, the Prewitt operator is used to calculate the derivatives in the x and y directions, respectively. Second, according to the algorithm, the square of the gradient is calculated, and the Gaussian method is used to reduce the noise. Finally, the Plessey operator value is calculated.

Through relevant operations, diagonal points are processed accordingly. The pixel point in the first image is set to x, and the window with size ( 2 m + 1 ) × ( 2 n + 1 ) centered on x is used as the contrast window. In the second image, the correlation operation is performed in the search area of size ( 2 d u + 1 ) × ( 2 d b + 1 ) around the pixel. Related operations can be defined as:

The gray mean, standard deviation, and covariance of the correlation operation window at point x = ( u , b ) are

When the value of Cov ( x , x ′ ) is −1, it means that the two regions are completely different, and when it is 1, it means that the two regions are completely the same.

A relaxation algorithm is used for matching optimization. The basic principle of this method is as follows: assuming that ( x , x ′ ) is a candidate matching point, M ( x ) and M ( x ′ ) are adjacent regions with a radius of R and a center of x and x ′ , respectively. If ( x , x ′ ) is a good match point, there are many match points ( y , y ′ ) . Among them, y ∈ M ( x ) and y ′ ∈ M ( x ′ ) , or there are few matching points, or even no matching points.

In practice, due to the large number of feature points extracted in the low-level processing stage and the large amount of computation, it takes a long time to carry out the corresponding processing. In order to solve this problem, the threshold limit method can be used to select better matching points while maintaining certain feature points. However, the number of reserved feature points is difficult to calculate. If the retention rate is too high, the matching time cannot be effectively reduced. On the contrary, the accuracy of matching is greatly affected, and the corresponding image may not be found.

In addition, in order to speed up matching, some scholars suggest that feature extraction should be carried out before matching. The characteristics and objectives of this method are the same. It is to shorten the matching area by extracting feature points, thus speeding up the matching speed and improving the matching accuracy. However, if too few feature points are extracted, accuracy cannot be guaranteed, which is also a big problem.

Therefore, in order to improve the corresponding image processing speed and ensure that the corresponding image processing speed has strong robustness, more efficient feature extraction technology must be adopted to solve the aforementioned problems.

In this article, an edge detection algorithm based on image pixel weight is proposed to achieve fast panorama matching.

The edge point set extraction algorithm is as follows:

It is assumed that the resolution of the image is N × M and the weight of each pixel is u nm = 0 . The image is updated according to the weight of 8 adjacent pixels:

Among them, O ¯ o = 1 9 ∑ l = o − 1 o + 1 ∑ z = k − 1 k + 1 u l z represents the average intensity of the area adjacent to the pixel. Each pixel is detected. When the weight exceeds a certain threshold, the region is determined as a boundary pixel, and vice versa. The noise is reduced. If the weight of the measured pixel is non-zero, and the weight of its eight neighboring pixels is 0, its weight is 0.

The fast point correspondence algorithm is as follows.

Plessey’s corner extraction method is used to extract the corners of two images and obtain the set of feature points. The edges of the two images are extracted, and the set of boundary points is obtained. For the points in the point set, the area correlation method is used to establish a preliminary point correspondence. The relaxation algorithm is adopted to relax the corresponding points and further find out the corresponding relationship. By limiting conditions, corresponding points that do not meet the conditions are further filtered.

The image modeling algorithm is designed in combination with the environment of AR and VR technologies, which can transfer data to the server. After receiving the information, the data are analyzed, displayed, and stored, and the relevant design of the algorithm model is completed in combination with the system architecture to test the system. The image modeling algorithm model of environment design based on AR and VR technologies is shown in Figure 4.

Figure 4

Environment design image modeling algorithm model.

It can be seen from Figure 4 that the image modeling algorithm model designed in the AR and VR environment is mainly composed of three modules: visual interaction module, entity information acquisition module, and rendering system. The model is user-centric. The visual interaction module collects data through sensors, cameras, VR glasses, and other devices. The entity information acquisition module collects images through the graphics rendering system. The rendering system completes the rendering of images through three-dimensional (3D) reconstruction technology [27].

3.1 Algorithm verification

In this study, different objects in the five environmental designs of object A, object B, object C, object D, and object E were selected as the environmental design content to be modeled, and five environments composed of at least two objects, environment 1, environment 2, environment 3, environment 4, and environment 5 were selected as the environmental design to be modeled. This experimental algorithm verification is divided into three categories: time verification, authenticity verification, and interactivity verification. For time verification, the experimental settings were designed by selecting different environmental objects and environments, and modeling using traditional image modeling methods and proposed image modeling methods, and the modeling of each method under different objects and environments was recorded time. The experimental method uses a timing tool to record the start and end time of each modeling and calculates the time used for modeling. Multiple experiments are conducted for each method on different objects and environments, and the average value is taken as the final result. The authenticity verification experiment also selected different environmental design objects and environments, and used traditional image modeling methods and proposed image modeling methods for modeling, recording the authenticity of the modeling results of each method. The experimental method evaluates the authenticity of the modeling results by comparing them with the actual environment, using objective evaluation criteria and including expert review. The interactive verification experiment settings are the same as the above two verification methods. The experimental method uses a questionnaire to evaluate the user’s interactive experience with the modeling results, including the user’s operational feedback and evaluation of the ease of use of the interactive interface. The method proposed in this study has been tested and verified. In order to test the correctness and feasibility of the whole algorithm, a virtual environment with certain regular shapes and virtual scenes was constructed using the geometric model based on the graphics model, and the required real-time performance was achieved.

Figure 5 shows the time-related experimental data of image modeling for different objects and different methods in different environments.

Figure 5

Image modeling time for different objects and different methods: (a) object and (b) environment.

It can be seen from Figure 5a that when facing object A, the modeling time values of the traditional image modeling method and the image modeling method proposed in this article were 12.14 and 8.72 s, respectively. When facing object B, the modeling time values of the two methods were 9.68 and 4.11 s, respectively. When facing object C, the modeling time values of the two methods were 17.75 and 11.92 s, respectively. When facing object D, the modeling time values of the two methods were 18.09 and 12.24 s, respectively. When facing object E, the modeling time values of the two methods were 14.67 and 9.86 s, respectively.

It can be seen from Figure 5b that when facing environment 1, the modeling time of the traditional image modeling method and the image modeling method proposed in this study were 75.16 and 42.20 s, respectively. When facing environment 2, the modeling time values of the two methods were 62.37 and 31.33 s, respectively. When facing environment 3, the modeling time values of the two methods were 89.15 and 46.98 s, respectively. When facing environment 4, the modeling time values of the two methods were 98.43 and 50.17 s, respectively. When facing environment 5, the modeling time values of the two methods were 79.12 and 44.54 s, respectively.

From the data in Figure 5, it can be seen that the modeling time of the image modeling method proposed in this study is better than that of the traditional image modeling method.

Figure 6 shows the experimental data related to the authenticity of image modeling for different objects and different methods in different environments.

Figure 6

The authenticity of image modeling with different methods under different objects and environments: (a) different objects and (b) different environments.

It can be seen from Figure 6a that when facing object A, the modeling authenticity of the traditional image modeling method and the image modeling method proposed in this study reached 87.51 and 98.57%, respectively. When facing object B, the modeling authenticity of the two methods reached 90.36 and 98.89%, respectively. When facing object C, the modeling authenticity of the two methods reached 82.62 and 96.28%, respectively. When facing object D, the modeling authenticity of the two methods reached 81.44 and 95.91%, respectively. When facing object E, the modeling authenticity of the two methods reached 86.63 and 97.77%, respectively.

It can be seen from Figure 6b that when facing environment 1, the modeling authenticity of the traditional image modeling method and the image modeling method proposed in this study reached 86.42 and 97.25%, respectively. When faced with environment 2, the modeling authenticity of the two methods reached 88.37 and 98.13%, respectively. When faced with environment 3, the modeling authenticity of the two methods reached 78.84 and 95.84%, respectively. When faced with environment 4, the modeling authenticity of the two methods reached 82.23 and 94.66%, respectively. When faced with environment 5, the modeling authenticity of the two methods reached 80.51 and 96.58%, respectively.

From the data in Figure 6, it can be seen that the modeling authenticity of the image modeling method proposed in this study is better than the traditional image modeling method.

Figure 7 shows the experimental data related to the interactivity of image modeling for different objects and different methods in different environments.

Figure 7

Interactivity of image modeling with different methods under different objects and environments: (a) object data and (b) environmental data.

It can be seen from Figure 7a that when facing object A, the modeling interactivity of the traditional image modeling method and the image modeling method proposed in this study reached 79.14 and 96.45%, respectively. When facing object B, the interactivity of the two methods reached 76.21 and 95.18%, respectively. When facing object C, the interactivity of the two methods was 80.98 and 98.35%, respectively. When facing object D, the interactivity of the two methods was 82.94 and 98.92%, respectively. When facing the object E, the interactivity of the two methods reached 80.35 and 97.12%, respectively.

It can be seen from Figure 7b that when faced with environment 1, the modeling interactivity of the traditional image modeling method and the image modeling method proposed in this study reached 77.61 and 98.06%, respectively. When facing environment 2, the interactivity of the two methods reached 75.15 and 98.61%, respectively. When facing environment 3, the interactivity of the two methods reached 79.57 and 96.32%, respectively. When facing environment 4, the interactivity of the two methods reached 81.38 and 95.68%, respectively. When faced with environment 5, the interactivity of the two methods reached 79.16 and 97.56%, respectively.

From the data in Figure 7, it can be seen that the interactivity of the image modeling method proposed in this study is better than the traditional image modeling method.

In order to better compare the performance differences between the experimental method in this article and the traditional method, object A is now taken as an example, and the comparison is shown in Table 1. It can also be seen from Table 1 that the interactivity of the image modeling method proposed in this article is better than the traditional image modeling method.

One of the reasons why the proposed method performs better in terms of authenticity is that it adopts a modeling approach based on multi-view photo sequences. By collecting photos from different angles and combining them, one can more fully capture the details and features of a target object or scene. This multi-perspective data collection method can better reflect the appearance and form of objects in the real world, thereby improving the authenticity of modeling. Another reason is that the proposed method takes advantage of AR and VR technologies, which can provide a more immersive experience. By displaying and interacting with real-world objects and scenes in a virtual environment, users can feel the authenticity of the modeling results more intuitively. For example, in environmental design, users can roam in the modeled environment and interact with virtual objects through AR and VR technologies, which can enhance their perception of the authenticity of the modeling results.

One of the reasons why the proposed method performs better in terms of interactivity is that it adopts an image modeling algorithm model for environment design based on AR and VR technologies. This model allows users to directly interact with modeling objects in the virtual environment, allowing users to interact with modeling results more naturally and intuitively. In comparison, traditional image modeling methods may be more static and limited, and users cannot directly interact with the modeled objects as they can with AR and VR technologies. Another reason is that the proposed method incorporates real-time tracking technology of user behavior and location. By tracking a user’s behavior and location in real time and applying this information to virtual environments, a more personalized and interactive experience can be achieved. This gives users greater flexibility in interacting with modeled objects and the freedom to explore the virtual environment according to their needs.

In summary, the reasons why the proposed method performs better in terms of authenticity and interactivity include factors such as the modeling method of multi-view photo sequences, the utilization of AR and VR technologies, and real-time tracking of user behavior and location. These factors work together to enable the proposed approach to provide a more realistic and interactive modeling experience.

3.2 System model verification

Based on the algorithm verification above, this study verified and analyzed the whole image modeling algorithm system model of environment design based on AR and VR technologies. Stability testing evaluates the stability of the system under different conditions by running the system for a long time or processing a large amount of data. The indicators include the system error rate for display. The operation fluency test evaluates the fluency and response speed of the system during operation and uses user experience and system performance indicators to evaluate. The indicators are expressed by the system freeze rate. Performance loss assessment evaluates the performance of the system under different loads, including computing resource utilization, memory usage, power consumption, etc. The indicators are expressed by the performance degradation percentage of the system.

Table 2 shows the data related to the stability test of the system model.

It can be seen from Table 2 that when the number of image modeling in the environment design was 2, the stability of the system was 99.37%. When the number of image modeling in the environment design was 4, the stability of the system was 98.66%. When the number of image modeling in the environment design was 6, the stability of the system was 98.05%. When the number of image modeling in the environment design was 8, the stability of the system was 97.24%. When the number of image modeling in the environment design was 10, the stability of the system was 96.80%.

Table 3 shows the data related to the running smoothness test of the system model.

It can be seen from Table 3 that when the number of image modeling in the environment design was 2, the system’s running smoothness was 99.25%. When the number of image modeling in the environment design was 4, the smoothness of the system was 98.48%. When the number of image modeling in the environment design was 6, the smoothness of the system was 97.61%. When the number of image modeling in the environment design was 8, the smoothness of the system was 97.02%. When the number of image modeling in the environment design was 10, the smoothness of the system was 96.23%.

Table 4 shows the data related to the performance loss test of the system model.

It can be seen from Table 4 that when the number of image modeling in the environment design was 2, the performance loss of the system was 0.75%. When the number of image modeling in the environment design was 4, the performance loss of the system was 1.64%. When the number of image modeling in the environment design was 6, the performance loss of the system was 2.31%. When the number of image modeling in the environment design was 8, the performance loss of the system was 2.45%. When the number of image modeling in the environment design was 10, the performance loss of the system was 4.67%.

To sum up, this study tested the image modeling algorithm system model of environment design based on AR and VR technologies from three aspects: stability, running smoothness, and performance loss. The test results show that the system model has excellent performance in image modeling.

For texture-based methods, the following challenges may be encountered.

1. Accuracy of texture extraction and mapping: Ensure that the texture extracted from the image can be accurately mapped into the reconstructed surface model to maintain the authenticity and accuracy of the model. The strategy is to develop efficient texture extraction and mapping algorithms to increase speed and accuracy and minimize the need for computational resources.

2. Noise and distortion processing: In the image modeling process, the noise and distortion that may exist in the image must be taken into account to avoid the impact of these factors on texture extraction and mapping. Use image processing techniques to reduce the effects of noise and distortion, such as using filters and denoising algorithms.

3. Speed and efficiency: Ensure that the texture extraction and mapping process has sufficient speed and efficiency to achieve real-time or near-real-time image modeling and rendering. Leverage hardware acceleration technologies such as modern graphics processing units (GPUs) to increase the speed and efficiency of texture processing.

4. Algorithm complexity and resource requirements: Ensure that texture-based methods have appropriate algorithm complexity and resource requirements to make them suitable for different types of hardware platforms and application scenarios. Use deep learning and machine learning methods to improve the accuracy and efficiency of texture extraction and mapping, such as using convolutional neural networks (CNN) to learn texture features.