AI Masks with Healing

(Posted 12/2/2022 as of LrC/12.0.1)

This blog relates to using one of the AI masking tools (Sky, Subject, background, etc.) in conjunction with one of the Healing tools.  When you do this, in some cases order matters and in some cases you will need to re-compute your AI Mask.  These interactions are somewhat complex but can be summarized in these 4 rules:

Rule 1 – When an AI mask is computed (either initially or re-computed) it uses the pixels as shown on the screen at that time, including any healing done prior to the mask being computed, to determine the area selected for the mask.

Rule 2 – Once created, the area selected by an AI mask is not altered unless you re-compute the mask. Sometimes this happens automatically, sometimes not.  For example you clone out a "Subject" after the Subject mask was created.  The area selected by the subject mask is still selected (and adjusted) by the subject mask even though the subject was cloned out and is no longer there

Rule 3 – If different content is placed into an area currently selected by an AI Mask, the Mask adjustments will be applied to those pixels as those places on the image are still selected by the mask.

Rule 4 – If Pixels are copied from an area selected by an AI mask to an area not selected by that mask, those pixels will lose the adjustments applied through the mask as they are no longer in an area selected by the mask.

Ghosts

One of the most common symptoms of when the above rules cause problems is “ghosts”.  This is where something that was removed still remains as a ghost of its former self.  Here’s a simple example

01 LR017 01 Ghost_

If you wind up with a similar problem, Re-compute your AI Mask See “Recompute Masks” near the bottom of this blog for info on how to re-recompute masks
 

CASE 1 – Healing an area selected by an AI Mask (Order matters)

If you need to do a Healing operation where the destination of the Heal is in an area selected by an AI mask the order matters

So, let’s start with the shot below where I want to remove the left boy and convert the dad and right boy to monochrome.

02 LR017 Case 1 A

In test one, I first create an AI mask (in this case a “people” mask for all 3 people) and de-saturate the 3 people. 

03 LR017 Case 1 B

Then I go to the Healing tool (any of the 3, but in this case I used “heal” mode) and painted over the left boy to have him replaced by ocean wave.  As you see, even though the source of the “Heal” was blue and white water, when those pixels arrived at the destination location they picked up the B&W from the People mask even though the idea was to replace that portion of the People mask with water.  Oops.

04 LR017 Case 1 C

Now let’s try it in the other order.  First I do the “Heal” to replace the boy with water. 

05 LR017 Case 1 D

Then I add the People Mask.  In this case the “people” mask (all people) only found the two remaining people and as such the de-saturation only affected those two – not the “healed” area
 

06 LR017 Case 1 E

CASE 2 – Using area selected by an AI mask as a source for Healing

When you use any of the Healing tools on an image where there is an AI mask and the source pixels of the healing operation come from an area selected by the AI mask, the pixels it uses as the source for the healing do not include the adjustments made on the AI mask unless they land in an area which is also part of that AI mask. 

So, starting with the same image, this time I created an AI mask for the Background (everything but the 3 people) and de-saturated it. 

07 LR017 Case 2 A

Then I went to the Healing tool and removed the left boy (I used clone mode, but any of the Healing modes would do the same thing). It did indeed remove the boy but as the place the left boy had been is not part of the “Background” mask it did not pick up the desaturation from the AI Mask adjustments (meaning it got color in this example).  Depending on the aggressiveness of the adjustments you made using the AI Mask, this problem may not be obvious. 

08 LR017 Case 2 B

If I now go back to the Masks tool and select the background AI mask and hover over the Background component (circled in red below), there is a pop up warning (which I can’t get a screen shot of) that says “Adjustments that may affect <component name> changed”.   This means that you should Re-compute the AI Mask which in this case would change the blue hole where the left boy had been was to monochrome.

09 LR017 Case 2 C

CASE 3 – Healing creates new subject matter

Case 3 is where an AI mask selected some content, then a Healing operation created new content that the AI mask would have selected had that content been there when the AI Mask was created.  In this example, the All people AI Mask works as expected and detects the 3 original people who I de-saturated. 

10 LR017 Case 3 A

Then I gave the left boy a twin brother by cloning a copy of him to his left.  As the 4^th person was not there when the AI Mask was created the Mask adjustments don’t include him and he’s in color even though the source of the clone was in monochrome.

11 LR017 Case 3 B

But if I re-compute the mask it finds all 4 and the 4^th person gets the AI Mask adjustments

12 LR017 Case 3 C

Re-compute Masks

Re-computing masks applies to masks containing AI selection algorithms such as Subject, Sky, Background, People, Etc.  Re-computing is required from time to time when something has changed which would cause the computed area of the mask to be different.  A classic example is if you copy an AI mask to another image where the “thing” (e.g. “the subject”) does not occupy the same place in the target image as it did in the image the mask is being copied from.  Another example is if you apply a preset containing an AI Mask as the selected items need to be re-found in the new image.  Re-compute forces LrC to run the AI logic on the image again to determine where the targeted item is in those images.

To re-compute an AI Mask, go to the Develop Module and click on the “Settings” menu.  This is not “Catalog Settings”, and the menu item only exists when you are in the Develop Module - or use <ctrl>+<alt>+U (Windows) or <Cmd>+<Option>+U (Mac).  Then select “Update AI Settings” and it will re-compute the active image. 

13 LR017 Recompute A

If you need to re-compute AI masks on multiple images, go to the Library Module, select the desired images and use menu path “Photo -> Develop Settings -> Update AI Settings” (short cut key combination <Ctrl><Alt>u (windows) or <Cmd><Opt>u (Mac). 

03 LR017 Recompute B

For multiple images you could also use the menu item in the Develop Module with Auto-Sync turned on to re-computed all selected images (not recommended).

Prior to LrC/11.4 when an AI mask needed to be re-computed, you got a warning message when you selected the mask needing to be re-computed (below).  To recompute the mask click the “Update" button below the message.

02 LR017 Recompute C

In LrC/12.0 there is no longer a significant warning message – only a flitting pop up message if you happen to hover your mouse over a Mask Component that may need to be re-computed.  But the warning message above returned in LrC/12.1 The warning message is at the top of the adjustments panel when An AI Mask needs to be re-computer AND you have the AI Mask Component selected or you have a Mask selected that only contains one component which happens to be an AI component that needs to be recomputer.  They say that AI masks are re-computed automatically in some, but not all, cases where it is needed. 

Below is an example where re-computing a mask is required.  In this image I created a “Background” AI Mask and lowered the saturation to B&W (2nd image).  I then used the healing tool in clone mode to loosely brush over the boy (3rd image).  Some of the destination area was already background and some of it was the boy.  LrC choose a section of water to use as the source of the clone.  The clone operation copied the pixels to the new location.  Those pixels that landed in an area which was part of the background mask got the adjustment from the mask.  However pixels that landed where the boy had been did not get the B&W adjustment from the background mask as that area was not part of the background when the mask was created.  In other words, the mask adjustment was applied to the pixels that landed in an area selected by the mask but not other areas (Image 3).  This is not what I wanted.

01 LR017 recompute D           

In order to fix this I re-computed the AI mask to make it decide again what is and isn’t “Background” (in this case).  This time it treated the cloned out boy as “Background” and that area got the saturation reduction applied through the Background mask.