# Affecting all possible 2color pixels, i.e: RG no B, RB no G, BG no R .. in AGAL1 fragment shader

**URL:** https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818
**Category:** Other
**Created:** [June 18, 2019, 2:42pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818 "2019-06-18T14:42:03Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![MacShrike](https://avatars.discourse-cdn.com/v2/letter/m/c0e974/32.png) [@MacShrike](https://community.haxe.org/u/MacShrike)
#### Post date: [June 18, 2019, 2:42pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818/1 "2019-06-18T14:42:03Z")

</div>

I managed it in AGAL2, now I’m trying to rewrite it in AGAL1.  
I’m grateful for any assistance since i’m stuck  
The shader will be applied to the entire screen. Some parts will contain gradients using only R,G or B

Case1: For selecting only R, G or B pixels i got it,

Case 2:its the combo’s that I can’t get to work. Only 1 R, G or B has to be zero the Remaining 2 have to be greater than 0.

What I want to do is subtract a certain value from the pixel if it is the right colour( in case2; colours) and add 1 in case it goes below zero.

Case1 works like:

- Set temp register values for the two colours you don’t want if greater than a fraction (0.003) [SGE]
- Add all x, y & z values (set or not to the temp register mentioned above)
- Set another register value if a constant register (value of 1) is greater then the addition [SGE]
- Multiply the result by the value you want to subtract (so nothing happens if the pixel isn’t selected)
- Subtract that from the appropriate sampled pixel colour
- set a register to 1 if the affected pixel is less than 0 [SLT]
- add the result to the right x, y or z register in the OC  
CODE:  
mov ft0, v0  
tex ft1, ft0, fs0\<2d, clamp, nearest, mipnone\>  
mov ft2.xyzw, fc0.yyyy  
sge ft2.yz, ft1.yz, fc0.ww  
seq ft2.x, ft1.x, fc0.y  
add ft2.xyz, ft2.xyz, ft1.xyz  
add ft2.x, ft2.x, ft2.y  
add ft2.x, ft2.x, ft2.z  
add ft2.x, ft2.x, fc0.w  
sge ft2.xyz, fc0.zzz, ft2.xxx  
mul ft2.x, ft2.x, fc1.y  
sub ft1.x, ft1.x, ft2.x  
slt ft2.x, ft1.x, fc0.y  
add ft1.x, ft1.x, ft2.x  
mov oc, ft1

So CASE 2:  
I thought for a double colour selection this might work, it doesn’t but I cant really see why not:

- Set registers to 1 from all XYZ greater than fraction (0.003)
- Multiply the one you want to exclude with 5 (constant register fc1.x)
- Add the fraction (0.003) to that to exclude register
- Add all set register X&Y&Z together.
- Set the colour registers you DO want to 1 if the addition \>= 2 [SGE]
- set the colour registers you DO want to 0 if the addition \> 5 (which would mean the unwanted colour has been seen) [SLT]
- and the same Mulitply this outcome with the value to subtract
- subtract the outcome of that multiplication
- set a register if result is \< 0 [SLT]
- add result of that SLT register to the OC registers of the two desired pixels

Alas it doesn’t seem to work.  
mov ft0, v0  
tex ft1, ft0, fs0\<2d, clamp, nearest, mipnone\>  
mov ft2.xyzw, fc0.yyyy  
mov ft2.xyzw, fc0.yyyy  
sge ft2.xyz, ft1.xyz, fc0.www  
mul ft2.x, ft2.x, fc1.x  
add ft2.x, ft2.x, fc0.w  
add ft2.x, ft2.x, ft2.y  
add ft2.x, ft2.x, ft2.z  
sge ft2.yz, ft2.xx, fc1.zz  
slt ft2.yz, ft2.xx, fc1.xx  
mul ft2.yz, ft2.yz, fc1.yy  
sub ft1.yz, ft1.yz, ft2.yz  
slt ft2.yz, ft1.yz, fc0.yy  
add ft1.yz, ft1.yz, ft2.yz  
mov oc, ft1`

Why doesn’t this work? Can someone assist me in getting this to run properly.  
Highest regards,  
Mac

---

<div class="post-metadata">

### Author: ![RobDangerous](https://community.haxe.org/user_avatar/community.haxe.org/robdangerous/32/55_2.png) [@RobDangerous](https://community.haxe.org/u/RobDangerous)
#### Post date: [June 18, 2019, 3:09pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818/2 "2019-06-18T15:09:28Z")

</div>

Didn’t even know that’s possible in a shader, usually there’s a color mask API for that (I suppose even stage3d has that). How did you do it in AGAL2?

---

<div class="post-metadata">

### Author: ![MacShrike](https://avatars.discourse-cdn.com/v2/letter/m/c0e974/32.png) [@MacShrike](https://community.haxe.org/u/MacShrike)
#### Post date: [June 18, 2019, 3:51pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818/3 "2019-06-18T15:51:03Z")

</div>

Hi,  
You can use nested ifl and ifg opcodes and test for. X,y or z values  
Like: fc1.w = 0.003 never test for zero  
Fc1.Y is a real between 0 → 1  
Fc1.x = 1  
Code:  
Mov ft0, v0  
tex ft1, ft0, fs0\<2d, clamp, nearest, mipnone\>  
ife ft1.x, fc1.w ← from here  
ife ft1.y, fc1.w  
ifg ft1.z, fc1.w ← to here select blue only pixels  
sub ft1.z, ft1.z, fc1.y \< subtract looping variable  
ifl ft1.z, fc1.w  
add ft1.z, ft1.z, fc1.x ← add 1 if \< 0  
eif  
eif  
eif  
eif  
mov oc, ft1 ← move to the output

---

<div class="post-metadata">

### Author: ![RobDangerous](https://community.haxe.org/user_avatar/community.haxe.org/robdangerous/32/55_2.png) [@RobDangerous](https://community.haxe.org/u/RobDangerous)
#### Post date: [June 18, 2019, 4:15pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818/4 "2019-06-18T16:15:27Z")

</div>

Ah, I didn’t understand that correctly initially. It’s not about only affecting two color components in the pixel you are writing but about extracting only two color components from a texture, right?

---

<div class="post-metadata">

### Author: ![MacShrike](https://avatars.discourse-cdn.com/v2/letter/m/c0e974/32.png) [@MacShrike](https://community.haxe.org/u/MacShrike)
#### Post date: [June 18, 2019, 7:21pm UTC](https://community.haxe.org/t/affecting-all-possible-2color-pixels-i-e-rg-no-b-rb-no-g-bg-no-r-in-agal1-fragment-shader/1818/5 "2019-06-18T19:21:29Z")

</div>

Indeed. I wish to know if two of the colour values in a texture pixel have a value \> 0.003 and the other one only zero/very low \< 0.003.  
Then subtract a constant register value from the two registers that have values.
