Featured Image for Watch virtual Pokemon interact with the real world using Niantic’s amazing new tech

Watch virtual Pokemon interact with the real world using Niantic’s amazing new tech

For a little while there in 2016, it looked like Pokemon Go – Niantic’s groundbreaking piece of AR – was going to take over the world of gaming.

It got a run at Splendour in the Grass, was banned in Iran and receieved the worst attempt at a comedic spin-off you’ve ever seen (I’ll never get those three-and-a-half minutes back).

It was thought that beacons would become huge selling points for shops and, to be fair, there were nights where entire neighbourhoods were overrun by players because a rare Pokemon had been spotted in the area.

Then, I don’t know, we all sobered up and just kinda moved on with our lives.

Well, Niantic hasn’t given up the ghost, continuing to push forward with AR – which is still a viable tech that they are totally right to be investing in – and using characters from Pokemon to illustrate its effectiveness.

In a video posted at the end of June, Niantic showed how they are using neural networks to perform occlusions in AR.

Or, to be a bit more down-the-line, how they can now make AR interact with the real world.

The video initially shows Pikachu in standard AR, whereby the little dude hops around in ‘the real world’ but is just sort of superimposed over the top of it – bouncing over objects and people.

By contrast, the occlusion technology sees Pikachu seemingly able to register when and where there are physical objects and people in the real world, and subsequently move in, around and behind them, as well as make the decision to get out of a passing human’s way.

The technology was developed over a number of years by Matrix Mill – a team of computer vision and machine learning experts from University College London.

Niantic acquired the team on June 28, the same day that the above video – which is introduced by Michael Firman of Matrix Mill, who refers to himself as “Michael from Niantic” – was posted to YouTube.

“The Matrix Mill team has come up with novel ideas that push the boundaries of what machines can process, thinking around occlusions, and seeing the world closer to the way human eyes can,” Niantic said on their blog when announcing the acquisition.

“As a result of this hard work, AR experiences can feel more natural to the eye, which is a goal we have squarely in our sights.”

About the author

Joe was Junior Vice-President at Compu-Global-Hyper-Mega-Net until it was bought out by Bill Gates. He now subedits for Conversant Media and considers it a step up.

Leave a comment