@NorbertThoden thanks for the update. I do have some more news from our end as well. Looks like the tiff/GeoTiff workflow is a limitation of the format itself rather than an Esri bug. The mask function should get you what you need, but also has some limitations.
Unfortunately, for GeoTIFF/TIFF, there's no well-defined way of supporting transparency channel/band. This isn't an Esri bug, it's related to this particular format.
In this case, I would recommend going with PNG route, which does support transparency in band 4.
The mask function would only allow you define a pixel as ON or OFF, will not give you transparency, i.e. percentage of "opaqueness".
For their test data, looks like they just need a way to set NoData, not percentage of transparency, so in theory, you could define a mask using band 4, but doing so would involve extra step, it may not perform as fast as PNG.