#what is realityOS technology | Explore Tumblr posts and blogs

tastydregs · 3 years ago

Text

Cevat Yerli created Room to blend video conferencing with 3D-animated scenes

Interested in learning what's next for the gaming industry? Join gaming executives to discuss emerging parts of the industry this October at GamesBeat Summit Next. Register today.

Room is rolling out today as the latest collaboration tool combining video conferencing cameras with 3D-animated backgrounds. The social 3D communications engine was created by a team led by former Crytek game developer Cevat Yerli, who was behind the Far Cry, Crysis and Warface series of video games.

The founders plan rollout of Room (spelled ROOM by the company) is designed with the new reality of working in mind to make remote teamwork easier. The browser-based, one-click tool has been in open beta for creators, and a general rollout starting now.

Yerli is the latest game developer to try his hand at metaverse-like conferencing solutions. Last week, former VR game devs Alex Schwartz and Cy Wise of Absurd:joy launched their beta version of the collaboration tool Tangle. They were partly inspired by the success of game devs who created Slack and Discord. Both of those companies started out making games, but they both pivoted to make communications tools that have become extremely popular.

It uses video capture from your webcam to take your live capture and place you in a 3D animated scene. Your video is literally an overlay in a coffee shop or an auditorium or a theater. But what’s you’re seeing in the characters is live video.

The tool combines video capture from your webcam to take video of your face and upper body. Then it places you in a 3D-animated scene. It’s actually video of you, cut out and pasted into an animated background. The trick is making the blending appear to be seamless.

I asked for more explanation on how it works, and the company said, “Room has purposefully arranged the camera setting in a way that when sitting at the table and looking around, people are placed with a specific fob on the camera path (which Room calls ‘table view’) so that people naturally integrate with the 3D background and the 2D effect is not visible. You do not have to sit at the table to be captured. You can freely move around. However, Room can only show of people what their camera video is providing in real-time. The team is working on solutions for people who are walking around to be shown integrated into the 3D environment as well. This will be part of a future release and is still in R&D stage.”

The opportunity is to produce something that is more joyful for employees to use in the metaverse age than the standard Zoom, Google Meet, Discord or Microsoft Teams. Meta is among the companies using solutions such as VR to take conferencing into the metaverse.

Yerli’s company isn’t pushing Room as a metaverse tool to be used to build virtual worlds. Rather, it is a communications tool that can support the metaverse strategies of other companies. It uses your camera to capture your reactions and converts them into expressions that your avatar can make. It does not require browser extensions. Cevat will be in New York attending the Clinton Global Initiative and the Future Investment Initiative where he’ll be debuting the product.

Room’s broadcast space.

Luxembourg-based Room wants to make it easy to create lifelike or fantastical online meeting spaces using its proprietary social 3D communications engine RealityOS. It replaces video camera calls with blended video and 3D animations that use lightweight data transfer so the communication happens at 60 frames per second, Yerli said in an interview with GamesBeat. That makes interaction more natural.

As tech giants grapple with arrangements to return to the office, retaining company culture is at an impasse. About 51% of senior leaders worried that flexible work arrangements will make it difficult to maintain their organizations’ current culture, and the adoption of technology that supports and complements existing company cultures will be critical for future success, according to Mercer.

Research by INSEAD Knowledge found that a further 45% of workers said camaraderie and teamwork had declined since the start of the pandemic. Technology that bridges distance and creates a space for more uplifting and meaningful communication can result in better workdays and outcomes.

As part of the TMRW Foundation, founded by Yerli, Room uses real-time 3D, depth-of-field and real-time reflections that simulate a natural, first-person point-of-view where participants can freely move around the room and look at each other – just as they would in real life. The technology allows people to represent themselves realistically in 3D spaces and provides users with an experience that closely mirrors human nature, he said. Within the industry, Room features the highest level of social presence, Yerli said.

I tried out a demo of Room and it was pretty snappy. I didn’t see interaction delays like I normally do in video calls. When you look at the people in the room straight on, they look pretty realistic even though they are 3D animated. If you move to the side, you can see the characters are like cardboard cutouts. It would be more realistic if they were fully 3D animated, but this is how Room gets its speed.

Origins

A campfire scene in Room.

Yerli started thinking about the tech after his twins were born in 2011. For this application, the technology that Crytek created for 3D-animated games would not work. Most of the work has been done in the last three years.

“It requires a completely different architecture and a completely different approach,” he said. “It was all about how I would see the world through the eyes of my children when they grew up. It was about what that next internet would look like. Other people reference the metaverse. For me, it was about information first, then people, social media, and life. I saw it as an internet of life.”

The idea is to share your life and the moments of life.

“The context defines what kind of transaction you do. If it’s a classroom, there is a learning transaction. If it’s shopping, there is a shopping transaction. For that smallest unit of life, contextual moments are shared. I was wondering how I could do this over the internet.”

In contrast to his career in games, Yerli is pushing a mass market technology with Room, not a high-end 3D graphics solution for a limited number of users.

“With Crytek, I was trying to push the envelope of graphics with Nvidia, AMD and Intel. Now we are pushing the boundaries of what you can do within one click away within a few seconds,” Yerli said. “We want you to enter a space and create lifelong experiences. My vision is to upgrade 2D to 3D first before going to VR.”

This approach also uses less processing power and so it’s better for the planet, Palomino said. Yerli said he thinks this lightweight approach will work better than selling a new generation of VR hardware to people. Yerli favors a 3D standard dubbed glTF right now.

Right now, about 16 people can participate in a Room meeting. The company is trying to push that upward over time.

“My whole life I have been pushing the high end. One thing I have learned is it is the masses that win the battle. We are always upholding that bar that it has to run on four billion devices,” Yerli said. “If I wanted to run on 20 million devices, then I would run on native apps and have complex avatars. That was my previous way of pushing tech for a limited audience. We want to make sure that we empower as many people as possible to connect from as many locations as possible.”

Key features

Room runs on just about any device. It runs on the browser and links via a single click.

Room uses an immersive first-person perspective. It has real-time and AI-supported video presence in 3D gives users physical depth and dimension. Participants are shown in the same shared space, giving them a realistic sense of presence and togetherness without the need for additional VR equipment.

It also features designer rooms that give shared context to meetings. Users have a wide range of Room options, including a hip New York coworking space, a cozy campfire in the woods, a Mediterranean beachfront, a Hollywood-style talk show set, an ethereal cloud room, and more. Each space has been with the help of interior decorators Claire Davies and Marcin Luebecki.

Room runs on RealityOS, a first patent-protected social 3D communications platform. It is built with a new browser-first 3D game engine core. The aim is to reduce experience video call fatigue and let users interact with objects and their peers in a natural and learned way.

It also has privacy protection, as users enter Room with the visual surroundings of their homes in the background automatically removed. All meetings are end-to-end encrypted. It is browser-ready and device-agnostic so users can access the platform by clicking a link within a web browser on any computer with a microphone and camera. No additional software downloads or VR headsets are required.

“We believe that digital spaces should be shaped by real life and with the presence of actual people, not anonymous avatars,” said Yerli. “Human nature is programmed to interact with others in a certain way. We close and deepen relationships face-to-face in shared spaces most of the time.”

He added, ” When we meet online, we should do it in one room and with our actual selves. We know how to bring people together in digital spaces. We think that the concept of Room will profoundly change the way people get together in the next phase of the internet. Not as a replacement for real life – but as the second-best option. We call it The Internet of Life.”

Room can be used for fun meetings as well.

Stefanie Palomino, chief product officer of Room, said that creating the right culture and values remains important in the age of remote work.

“It is clear from the research that while remote working offers huge emotional and financial benefits to both organizations and individuals, without the right technology in place, there remains a significant risk to our culture of togetherness,” Palomino said. “Due to the recent and rapid virtualization of corporate interactions, we have pushed ourselves into these two-dimensional, flat spaces. Room has been designed to encourage people to interact in uplifting virtual 3D meetings, and ultimately support and enhance unique cultural norms.”

The Founders Plan features access to all rooms including the exclusive Founder Room, lifetime founder status and price, early access to upcoming features, unlimited meeting time, and more.

I wrote one of the first stories ever on Yerli and his brothers as they launched Far Cry. Yerli served as CEO of Crytek from 1999 to 2018. While the company thrived on creating memorable games with high-end 3D graphics — Far Cry, Crysis, Warface, and more — it had trouble keeping up with the financial requirements and development demands of modern gaming.

He founded The TMRW Foundation in 2017 with the mission to create, acquire, and propel ideas, digital technology, and applications of the next iteration of the internet that have a positive impact on society and the way we define presence, interact as people, and generate profits. It has a portfolio of 3D simulations, VR, AR, and AI-powered products.

Palomino was the cofounder of Red Lab, a boutique consultancy started in 2015 to support clients in interactive, smart digital events and communications. The largely self-funded company started in 2017 and it has about 100 people. It is unrelated to Crytek.

The memory palace

Room builds a memory palace so you can remember things better.

Each Room is set up like a Memory Palace, a visualization technique that goes back to the ancient Romans and Greeks. With it, you create 3D spaces around the cool moments that happen in life. The 2020 video game Twin Mirror used this technique to solve mysteries.

Yerli believes the Memory Palace approach of Room is unique among the plentiful competition.

“The technology for avatars is pretty far away from being real, inclusive, accessible, and scalable,” he said. “So we came up with a technique based on AI inferencing. We can provide a presence or social presence quality that is unseen before.”

Yerli thinks this technique is more humanizing while enabling it to run in a browser that is only a click away.

“For me, the vision of the next internet is not so much about VR, it’s not so much about avatars, because often VR has a threshold, a technical barrier. And gamers don’t approve yet. The barrier is there, and the scale is not there. Our philosophy has been how can we build a foundation that can be operated by or clicked on and entered by four billion people. It has to be running on every browser, on any device. It has to be the most inclusive, the lowest lightweight engine.”

Accessibility doesn’t mean it has to sacrifice quality, as Room adds reflection shadows and AI inferencing to make it more realistic.

“What we’re trying to do is humanize technology,” Yerli said.

GamesBeat's creed when covering the game industry is "where passion meets business." What does this mean? We want to tell you how the news matters to you -- not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it. Discover our Briefings.

0 notes

marketusme · 3 years ago

Text

Apple's AR/VR plans teased by reference to 'realityOS' in App Store code

It's a fair bet that Apple is building a mixed reality headset.

As per normal, Apple is saying nothing about future plans beyond CEO Tim Cook expressing interest in augmented and virtual reality. But all the signs are there, from the hires to the leaks and more recently, the possibility of a standalone operating system devoted to VR and AR: realityOS.

Some programmers spotted the term in Apple App Store upload logs, posted the code screens to Twitter, and then 9to5Mac picked up on it. The references have since been scrubbed from the code, which gives even more credence to the notion that it was a slip-up on Apple's part (or maybe one of their partners).

If realityOS is real, it's another indication that Apple is moving forward with VR/AR projects which most likely include dedicated hardware.

The introduction of almost every single new Apple product category has led to a bespoke OS. There's:

macOS for the Mac

iOS for the iPhone

watchOS for the Apple Watch

iPadOS for the iPad

tvOS for Apple TV

Naturally, an Apple MR (mixed reality) headset would get its own platform as well.

There's also been some conjecture that the realityOS, if it exists, would be based on iOS. It's a safe assumption since the headgear will need a power-sipping, mobile-friendly platform, one that splits the difference between the touch-friendly interface of the iPhone and the mostly-hands-off approach for watchOS.

If realityOS is built on iOS, though, Apple will have to decide between using its current A-series mobile CPUs found in the iPhone (A15 Bionic) or one of the new M-series chips it's adding to its desktops and laptops.

Whichever processor it uses, we imagine realityOS will respond to a combination of head movements, eye-tracking (for intention), and touch sensitivity on the headset itself.

It's also possible that the early headgear might not have an incredibly lightweight OS, since it could draw most of its computing power from a wirelessly connected iPhone.

Analysis: When could we see Apple's MR headset?

Rumors are currently split between us seeing Apple's first attempts at AR/VR headgear by WWDC 2022 and the technology slipping into 2023. This being a completely new category for Apple, we're certain they won't show us anything until it's both eye-popping and ready to demonstrate live at least to an online virtual audience.

And there is no real rush here. We're in the early days of the VR and AR revolution. The Metaverse is a crazy dream that Apple is happy to sit back and watch as competition and competitors chase it like so many lemmings running to a cliff.

Not that Apple isn't keeping an eye on all the activity. During last month's Apple Earnings call, Cook said this about the prospect of the Metaverse, "That’s a big question. We're a company in the business of innovation. Always exploring new and emerging innovation. This area is very interesting to us. We have 14K AR apps in the App Store. We see a lot of potential in the space..."

From a consumer perspective, Apple remains focused on augmented reality and all the ways its current hardware lineup can enable it (cameras, LiDAR sensors, and powerful onboard neural processing). It'll likely extend those capabilities as far as it can before officially unveiling anything like realityOS.

Apple MR headset news, leaks and what we want to see

Source Link Apple's AR/VR plans teased by reference to 'realityOS' in App Store code

#TechRadar - All the latest technology news

0 notes