The Metaverse technology stack encompasses a diverse array of technologies and components across multiple domains, including content creation and management, Metaverse development and coding platforms, security, simulation engines, communications, Artificial Intelligence (AI), cloud computing, and auxiliary technologies such as Geographic Information Systems (GIS) and the Internet of Things (IoT).

This paper provides an overview of the fundamental elements within each layer of the tech stack.

Content creation and management

This layer encompasses the tools and platforms for creating, sharing, and curating digital assets which can include anything from virtual fashion to virtual real estate. Practitioners can play a significant role in fostering creativity and ensuring content quality by developing user-friendly creation tools and AI-driven content recommendation systems.

3D modelling and animation tools: These software applications, including Blender, Maya, and 3ds Max, serve as essential tools for crafting intricate 3D assets, characters, and animations that form the core of the Metaverse's immersive experience.

Digital art and design tools: Tools such as those found in Adobe Creative Suite enable the creation of 2D assets and textures that are indispensable in constructing virtual environments.

Asset libraries: These repositories are dedicated to storing and efficiently managing a variety of 3D models, textures, and other reusable assets that are vital to Metaverse content creation. For instance, Unity Asset Store serves as a repository for game developers to access and share 3D models, textures, and scripts. Game developers can purchase or upload their assets for reuse.

Asset optimisation tools: These specialised tools are used to optimise 3D assets, to reduce file sizes and enhance overall performance within virtual environments. Tools such as Simplygon and SpeedTree focus on optimising 3D assets. In a virtual world, these tools can optimise high-polygon count 3D models for better performance while maintaining visual fidelity.

Content Management Systems (CMS): CMS platforms are used for the systematic organisation, categorisation, and distribution of digital assets within the Metaverse to facilitate seamless access and utilisation. Contentful and Strapi are examples of CMS platforms for organising digital assets within the Metaverse. These platforms allow content creators to easily manage and distribute digital content such as virtual clothing, accessories, or 3D art.

Simulation engines

Physics Engines: Software such as NVIDIA PhysX and Bullet Physics simulate real-world physics within the metaverse, enhancing the realism of the virtual environment.

AI-Driven Simulations: AI and machine learning algorithms are leveraged to create dynamic and realistic simulations of virtual environments, including behaviour modelling, which enhances user immersion.  Engines such as SpeechGraphics bring AI-powered Non-Player Characters (NPCs) into the 3D world to give it a layer of immersion and create a diverse experience rather than a scripted one, ensuring a unique experience each time a player enters the virtual space.  Nvidia ‘s Omniverse platform also provides a new simulation engine called Audio2Face which provides AI-generated facial expressions for 3D avatars.

• NCS recently made use of the Audio2face module from Nvidia Omniverse to animate a digital human for a digital concierge project.  The animation of the avatar was accomplished by simply passing an audio stream through the module and letting Audio2face decide how to animate the avatar’s facial expressions

Communications

• Effective communication is essential in the Metaverse. This layer includes technologies for voice and text chat, virtual meetings, and social interaction. Practitioners should stay updated on platforms like VRChat, Horizon Workrooms, and emerging protocols for cross-platform communication.
• Voice and video chat: Real-time voice and video communication tools are integrated to facilitate social interactions within the Metaverse, enabling users to engage in meaningful conversations.
• Text chat: In-game text chat and messaging systems provide avenues for both one-on-one and group communication, enhancing the social aspects of metaverse experiences.
• Social media integration: Metaverse experiences are linked to social media platforms, allowing users to share their experiences and connect with a broader audience.

Metaverse development/coding platforms

These platforms serve as the technological bedrock upon which the Metaverse is constructed. They provide the tools, frameworks, and environments for developers to create, build, and innovate within the Metaverse.
Game engines: Engines such as Unity and Unreal Engine have been widely adopted for Metaverse application development due to their capabilities in handling complex 3D environments and interactive elements. 

• NCS has developed a project with a web-optimised games engine, Playcanvas, to create an engagement platform for a banking customer, serving both internal and external stakeholders.

Web-based frameworks: Frameworks such as WebVR, A-Frame, and Babylon.js enable web-based Metaverse development and ensure accessibility via web browsers. 

• NCS has used FrameVR to deliver ad hoc projects, including a virtual town hall for a banking customer and an annual event for a networking client. Web-based framework tools that support many users and allow for 3D editing online enable NCS to develop and deploy solutions quickly and easily.

AR/VR development kits: Software Development Kits (SDKs) such as Oculus SDK and ARKit/ARCore are instrumental for Augmented Reality (AR) and Virtual Reality (VR) Metaverse development.

• NCS has built immersive digital twins and sandboxes using these frameworks for investigation and operational uses cases that require Extended Reality (XR) simulations. Those solutions make use of Hololens and Oculus to provide an immersive view of an incident area and allow users to use hand gestures to manipulate the environment, to issue commands, and to gather data.

Artificial Intelligence (AI)

Artificial intelligence and automation play a crucial role in shaping user experiences and managing the vastness of the Metaverse. AI-powered virtual assistants, content moderation, and predictive algorithms enhance engagement and safety within virtual worlds.

AI chatbots: AI-driven chatbots serve as NPCs and virtual assistants within the Metaverse, helping and enhancing user engagement.

Behaviour modelling: AI is utilised to simulate lifelike behaviours and the interactions of virtual entities, significantly augmenting the immersive quality of the Metaverse.

Speech synthesis: Creates natural sounding voices for NPCs and for any form of narration that is key to the overall experience for users. Fortunately, there are many solutions in the market to   enhance the naturalness of speech, and the latest trend is to clone the voices of celebrities and other influential people to lend credence to narratives within the Metaverse environment. Creating a voice clone with just 45 seconds of real audio recording is already possible using platforms such as PlayHT.

Meta-spatial analysis: Users interacting within the Metaverse space present new data collection points (e.g., where users are located in the Metaverse and how users are navigating the space). These data sets allow content designers to strategically position content to achieve the best user engagement outcomes.

Secondary technologies (GIS and IoT)

The Metaverse's success depends on interoperability between platforms and technologies. Open standards and protocols are vital to ensure seamless cross-platform experiences. Practitioners should advocate for, and contribute to, the development of these standards.

GIS integration: Geographic data from GIS systems is incorporated into the Metaverse to provide location-based experiences and navigation capabilities.

IoT integration: IoT devices are connected to the Metaverse, enabling real-time data collection and interaction, including smart home integration and environmental sensor data acquisition.

Security

Security in the Metaverse presents a unique set of challenges as it needs to combine game-like features that traditionally prioritise speed and responsiveness over security. Fortunately, there are already existing technologies in place to meet these challenges.

Anti-cheat: This is a unique mechanism that governs behaviours and interactions of all players in the Metaverse environment.  This is especially important when there are rewards and achievements that tie in with some monetary value.  Having a robust anti-cheat mechanism in place becomes extremely important to ensure fair play.

Blockchain platforms: Platforms such as Ethereum, Flow, and Binance Smart Chain are leveraged for the development of blockchain-based Metaverse applications and the establishment of digital asset ownership.

Cloud computing

Cloud computing is crucial to the Metaverse as it provides scalable and cost-effective infrastructure for processing vast amounts of data, hosting virtual environments, and powering immersive experiences.

Server hosting: Leading cloud providers such as AWS (Amazon Web Services), Azure, and Google Cloud offer server infrastructure to host Metaverse services and applications, ensuring scalability and reliability.

Content delivery: Content Delivery Networks (CDNs) guarantee low-latency content delivery to users across the globe and contribute to a seamless user experience.

Additional security

As the Metaverse expands, security and privacy concerns intensify. Practitioners must prioritise cybersecurity, including identity protection, data encryption, and anti-phishing measures. Awareness of privacy regulations and ethical considerations is equally important.

Encryption: Encryption protocols are employed to safeguard data transfer and communications within virtual environments, protecting sensitive information from unauthorised access. Using encryption protocols, such as SSL/TLS to secure data transfer and communications within virtual environments is crucial for security. For example, SSL encryption ensures that user information remains confidential during in-game chats or with financial transactions. 

• NCS implemented a Metaverse platform that uses Colyseus to handle multiplayer networking, which works on UDP, and is inherently insecure.  But through a unique architecture design, we created a secured connection to the customer’s secure backend while still allowing a gamified frontend.

Access control and authentication: Identity verification and access control mechanisms are implemented to ensure that only authorised users can access specific areas or perform designated actions within the Metaverse.

Cybersecurity tools: Utilising a range of cybersecurity tools and best practices is essential to defend against potential threats such as hacking, data breaches, and other cyberattacks.

Conclusion

The Metaverse technology stack is a dynamic and continuously evolving ecosystem. Successful Metaverse development often requires a multidisciplinary approach, bringing together expertise in various domains to create immersive and interconnected virtual worlds. Going forward, interoperability and adherence to standards will remain pivotal to ensuring the seamless integration of differing Metaverse platforms and experiences.