Not sure what your confusion is? The screenshot shows a massive area included in the output (the big black area) that is outside the environment raster mask (outlined in red)... I have used a raster mask many times with kernel density successfully... I'm asking for information why the output would extend beyond the mask?