Camera Mapping

rustybroomhandle

I have been experimenting with a technique called camera mapping for our game, and have put together a test scene. Camera mapping is when you map a 2D scene onto 3D geometry by projecting your UV map from the camera view. This allows you to have light sources, reflections etc, that interact with your 2D scene as if it were 3D. It works best for fixed camera views or orthographic projection.

It's a not a super-difficult thing to achieve. I used Blender, but I'm sure it can just as easily be done in other packages.

My test scene: http://screwylightbulb.com/pj (36MB - Unity3D web)

This scene was painted in Photoshop and I applied some simple geometry to it so that the floor is on a horisontal plane and the buildings are all on upright textures. I added a reflective surface to the ground plane as well as raindrop animations. Lights will now also work correctly with the scene. It works with as much or as little geometric detail as you want.

From a different angle:



An example of one made using a stock photograph (tutorial available):

PowerCat

That's really awesome :)

rustybroomhandle

Oh and if anyone has a painted/drawn scene they'd like to have test-mapped into this format just to see how it might be applied in an actual game, let me know.

I'm dying to make a classic-style point & click adventure like this. :P

Stray_Train

Please make it a thing? Came across something similar once and the idea really excites me

creative630

I assume this would be constrained to how you split up the geometry? It just seems like this would work really well for background scenes without much noise, like it would break if you tried to separate a tree from a sky background? (in terms of actually cropping the tree out)

It seems really similar to the stuff done in point cloud scene reconstruction. But I guess the lighting would probably not work as well if you created a point cloud instead?


Very cool idea.

Tuism

Holy cow that is smart.

rustybroomhandle

creative630 said:

I assume this would be constrained to how you split up the geometry? It just seems like this would work really well for background scenes without much noise, like it would break if you tried to separate a tree from a sky background? (in terms of actually cropping the tree out)

Yeah I imagine for something like a tree you'd have a hard time separating bits out. Another tricky thing is that you have to account for parrallax when you have a moving camera by making sure that elements that are behind others are still properly textured.

In that stock photograph example I linked above, the guy had to use the clone tool and cut/paste quite liberally to fill in the blanks.

creative630

@rustybroomhandle Yeah that makes sense, I was wondering what he used to fill in those areas. In general I guess an artist could just manually add in the needed detail themselves (as laborious as it might be). Cloning/filling in photo-shop might not always be the best way to solve it :P

iceblademush

There is also this cool app for IOS and Android, it works pretty well:
You need about 20-30 photos to create a model.

http://www.123dapp.com/catch

Chris_Bischoff

A Whispered World 2 uses camera Projection to get some incredible results.





Also Pillars Of Eternity and Planescape Torment.

I love camera mapping - its certainly something I'm excited about exploring in the future!

rustybroomhandle

Chris_Bischoff said:

A Whispered World 2 uses camera Projection to get some incredible results.
---8<-- snip ---8<--
I love camera mapping - its certainly something I'm excited about exploring in the future!

I'm a bit sad that Whispered World 2 has gone this route though as it's so different from the first, which used more traditional techniques. Does look pretty good though.

There's a variant on this that I am experimenting with, for use with orthographic projection, ie no parallax. The workflow is something like this:

- Create 3D geometry for scene, mainly colours, no/basic textures
- Snap a camera view of the scene in very high resolution, give to 2D artist
- 2D artist paints over the scene, only drawing in shading for static light sources, or ignoring lights/shading altogether
- In Unity/other engine - project this painting onto the scene's geometry directly from the camera, but make the projector ignore characters and other movable objects
- Add lights, reflective surfaces etc.

- You end up with a beautiful painted scene that's still properly 3D - with all the benefits that provides

Pain in the butt to make changes though. 2D artist must use layers liberally so that "remove the couch" does not become a week's work.

Pomb

I'm experimenting with this technique for our trailers, It's pretty darn sweet. My workflow is as follows:
- Paint something rough
- Separate into layers for parallax.
- Project each layer on their own planes and start adding points to create form inside blender.
- Setup camera move.
- Clean up painting and fill in details where the parallax shows though.

This way I don't spend time painting things that aren't seen, but also means it's less kind to iterations on the camera move.

A Whispered World 2 looks amazing, thanks for posting really cool reference.

so that "remove the couch" does not become a week's work.

@rustybroomhandle it happened, and now layers are in abundance.