Unity shaders question

Denzil

Hi

I am working on improving the visibility and lighting in my game, and my shaders knowledge is a little lacking. For the visibility, I have taken an image of a white circle on a black background, and cut bits out where the player cannot see. I would've preferred it if those cut out bits were made black instead of cut out, but it's a package I bought with nightmarish code, that's a lot less straightforward than the problem sounds. I wanted to use this resulting image as a cutout mask, and pass destination pixels where the image is white, and just leave it black where it is not, with some simple blending. But now my geometry is wrong, and my mask is actually just gaps. Is it possible for me to, after I have rendered all my world geometry, to then render this cutout mask to a black "background" somewhere in memory (not screen), then render the result of this across my previously rendered world geometry.

I have considered doing the visibility first and setting the camera background to black. This initial bit works as expected, but if I then blend my lightsources in, they replace some visible regions with black, as the light source image has black around the edges like this:



I want to blend my light sources additively, and then render the world geometry onto that multiplicatively. But adding the visibility to this seems to be a mess. Not sure if I'm using the right terminology here, but I imagine it is what I want to do. If I simply add the light sources values, I get something like this:


Where the player can see a flashlight behind a wall. If I did it multiplicatively (starting with the visibility map rendered on top of a black background), I get this:


Now the black outline of the image shows, since it multiplied that black region with a part that should be visible. If you know anything that can help or can give advice, please do. My brain is melting.

PS. I have unity free so I can't use render textures if this matters.

Denzil

I manages to create a black and white visibility map like so:

I need to invert all the colours though, but nothing I try is working. I basically tried this:

Blend OneMinusDstColor one

With zwriting and lighting off. Just had that in a shader on a plane that I put in front of the camera. It's not inverting though. Just making everything black. Anyone know how to invert properly?

Chippit

What you want to do is complicated (but not impossible) in Unity Free.

Generally speaking, you have access to two separate extra channels of information that you can render to in Unity Free (using multiple cameras) that you can use in lieu of having render textures:

The first is the z-buffer. You can render invisible objects to it at various depths in order to make a mask for yourself, which can help for rendering light obstacles and such, if you so desire. By manipulating and reversing z-tests, you can actually do fairly complex stuff just with this.

The other is the alpha channel; rendering various values to it can also allow you to create all sorts of masks (but soft ones instead of hard ones like the z-buffer), and it can contain any information in it that you choose. Indeed, it may well be best for you to use it here for your light. If you never need the existing values in the alpha channel for any of your other rendering, you can render have all your shaders multiple by DstAlpha (and also make sure that you don't write INTO alpha), and then you can still get proper alpha blending in your rendering without nuking the colour values that you're currently using for your mask.

On your specific problems, it seems like you're going about things from the correct perspective, but it'll be difficult to track down your specific problems without more information: The exact render order of things (which cameras are drawing what, what their clear settings are, their draw order, etc).

However, for your second issue, there's a far easier way to go about doing that. Instead of drawing your obstacle mask white and then trying to invert it, it is probably much simpler to use subtractive blending for this instead. Have your camera clear to white, and draw all lights in black with subtractive blending. It should get you the expected result, if I understand what you want to achieve correctly.

Denzil

@Chippit thanks a lot for this! I appreciate it so much, I'd owe you a wookie life debt if I was a wookie.

I think for now I'll try to stick to the alpha channel.

Chippit said:

If you never need the existing values in the alpha channel for any of your other rendering

Luckily I don't. Or I don't think I do. I have some cutout shaders later at the end of my processing queue, but I believe this only uses the texture's alpha, and not the destination alpha (does this sound right?). I'll try what you suggested so I can entirely skip the inverting bit in the queue. It felt unnecessary when I did it, but I was kinda stuck :)

I have one question though, or perhaps a few. Firstly, how do I do this? ->

Chippit said:

you can render have all your shaders multiple by DstAlpha (and also make sure that you don't write INTO alpha)

There's two parts I am unsure of, the multiply, and then the making sure I don't write to the alpha channel. Currently I have something like this (for my flashlights):

Shader "2DVLS/LightFlash" {
    Properties 
    {
        _MainTex ("SelfIllum Color (RGB) Alpha (A)", 2D) = "white" {}
    }

    Category {
	
       Lighting Off
       ZWrite Off
       Fog { Mode Off }
      
       
       BindChannels {
              Bind "Color", color
              Bind "Vertex", vertex
              Bind "TexCoord", texcoord
       }
       
	   SubShader 
	   {
            Tags { "Queue" = "Geometry+7" }
 			
 			                                          
            Pass 
	     {

               Blend DstAlpha zero
     
               SetTexture [_MainTex] 
	       {
                    Combine texture * primary
               }    
            }
     
        } 
    }
}

In the pass I do

Blend DstAlpha zero

to try and multiply with the alpha channel but I really don't know if this is right. I just read what is on the Unity documentation, but don't really understand it too well.

Also, I don't understand

Combine texture * primary

. I've read up on this combine keyword a bit, but still feeling lost.

Thirdly, I changed the _MainTex property to this

_MainTex ("Texture 0", 2D) = "white" {}

quite arbitrarily. The shader I started off wasn't written by me, I'm just modifying it, and I wanted to make sure I'm not writing to the alpha channel, but not sure how.

Lastly, I also actually want my lights to add additively if this is possible. I want something like (Source Color + Destination Color) * Destination alpha, if that makes sense. The lights should blend together, adding the brighter components from each light to the end result, then multiply it with the alpha channel which is the "visibility mask". Is this doable?

Thanks again.

Chippit

Forgive the terse answer, I don't have much time right now to be thorough; I hope I answer all your questions anyway!

Denzil said:

There's two parts I am unsure of, the multiply, and then the making sure I don't write to the alpha channel:

So this approach

Blend DstAlpha zero

is correct to multiply outgoing pixels by alpha, though you could also do

Blend DstAlpha OneMinusSrcAlpha

so that multiple lights will properly blend together. For this to work you'll have to premultiply the outgoing pixel by alpha (assuming you're actually using the alpha as transparency in your light textures or whatever you're doing) in your shader, since the blend operation isn't doing it as normal. If you're just darkening the colour, then that won't be necessary.

This won't necessarily cause you to not write into the alpha channel though. For that you want to add:

ColorMask RGB

Denzil said:

Also, I don't understand

Combine texture * primary

. I've read up on this combine keyword a bit, but still feeling lost.

This is code for writing shaders using the fixed function pipeline, and I admit I have little experience with it since I never use it that way - however, to my understanding, that is the simplest line that will cause both vertex lighting and vertex colours to be applied to your output pixel.

Also,

Chippit said:

you can render have all your shaders multiple by DstAlpha

Wow. Sunday proofreading fail. :/

Denzil

Cool thanks a lot :) I actually thought I had updated my answer to let you know I got it working, but I see my update never went through (sucky internet).

At the moment I am blending my lights like this:

Blend DstAlpha DstAlpha

I never understood how exactly this works, but I think it's kinda like filling in the equation
SrcValue * param1 + DstValue * param2
from Blen param1 param2

If that makes sense. And because I understand it now, I managed to get it working. Also managed to get rid of the inverting step thanks to your advise :)

The reason I blend like that is because I want to add all the lighting components together, and then finally multiply it with the alpha value in my mask.

Thanks again!